System and Method for Determining the Orientation of an Eye

ABSTRACT

In a device or a method for determining the direction of vision of an eye, a starting point or a final point of a light beam reflected by a part of the eye and detected by a detector system, or of a light beam projected by a projection system onto or into the eye two-dimensionally, describes a pattern of a scanning movement in the eye. The inventive method uses a displacement device that guides the center of the pattern of movement into the pupil or macula center of the eye, and a determination device that uses the pattern of movement of the scanning movement to determine the pupil center or macula center.

This application is a divisional of U.S. patent application Ser. No.13/300,691, filed Nov. 21, 2011, titled DEVICE AND METHOD FORDETERMINING THE ORIENTATION OF AN EYE, the priority of which is claimed,the entire disclosure of which is expressly incorporated herein byreference, which is a divisional of U.S. patent application Ser. No.10/551,443, also titled DEVICE AND METHOD FOR DETERMINING THEORIENTATION OF AN EYE, now U.S. Pat. No. 8,113,657, which is the U.S.national phase of international PCT application PCT/EP01/11634, filedOct. 8, 2001, the priority of which is claimed, which claims the benefitof the filing dates of prior foreign applications PCT/EP00/09840, filedin the European Patent Office (EPO) on Oct. 7, 2000, PCT/EP00/09843,filed in the EPO on Oct. 7, 2000, PCT/EP00/09841, filed in the EPO onOct. 7, 2000, PCT/EP00/09842, filed in the EPO on Oct. 7, 2000,PCT/EP01/05886, filed in the EPO on May 22, 2001, and 101 27 826.8,filed in Germany on Jun. 8, 2001, the benefit of which filing dates isalso claimed in the present application pursuant to 35 U.S.C. §119.

BACKGROUND OF THE INVENTION

1. Field of Invention

The present invention relates to a system and a method for determiningthe position and/or orientation, particularly the direction of vision,of an eye by the serial intercepting of a beam reflected by a part ofthe eye.

2. State of the Art

The knowledge of the momentary orientation of an eye is a necessaryprerequisite for a plurality of the most different applications:

In medicine, the treatment of defective vision, retinal detachments,macula degeneration, etc. requires the detection of voluntary and mainlyinvoluntary orientation changes of the eye in order to, for example, beable to appropriately cause a laser beam to follow in such a mannerthat, in each case, it impinges on the same point on the retina or movesalong a certain trajectory on the retina.

In psychology, the orientation of the eye permits diverse conclusions.Thus, for example, a sudden strong rotation indicates the beginning of agrand mal or a black-out. Furthermore, significantly more extensiveinformation can be obtained from the direction of vision of the eye,which can be determined, for example, by the mean perpendicular of thepupil. As an example, the recognition pattern of test persons whenviewing certain pictures can be analyzed in this manner.

In addition, the knowledge of the direction of vision offers thepossibility of using the information that indicates where the viewer ismomentarily looking in order to identify the fixed object; whether it isa certain menu item on a (virtual) video screen, a device to beactivated (light switch, etc.), a selected target of a rocket launcher,or the like.

Furthermore, the knowledge of the orientation and particularly of thedirection of vision of the eye makes it possible to, for example,project visual information into the eye, which information is correlatedwith the viewer's perceived picture of the environment or his virtualorientation (for example, when using virtual-reality spectacles or thelike), so that the pictures projected into the eye, for example, areseemingly resting in space or relative to an object, or the mergedpicture changes corresponding to the direction of the vision of the userof the virtual-reality spectacles.

Diverse devices and methods have therefore been developed for detectingthe orientation of the eye:

German Patent Document DE 196 31 414 A1 mentions sensors which are fixeddirectly to the eyeball. These naturally, on the one hand, represent aconsiderable danger to and impairment of the test person, and, on theother hand, can also not detect the direction of vision of the eye (forexample, in the case of strabismic persons) and, in addition, are quiteinaccurate relative to the often minimal involuntary movements of theeye.

Furthermore, the integral taking of the retina reflex picture by CCDcameras is mentioned, which, on the one hand, because of the highexposure, is suitable for the low-light reflex picture but, on the otherhand, comprises a very large quantity of data resulting in lowprocessing speeds as well as distortion problems.

The above-mentioned document therefore suggests a device where an imageof the environment reflected by the retina is serially scanned by meansof a scanning device, is electronically modified and subsequently isprojected back into the eye.

However, in this case, the orientation of the eye, particularly thedirection of vision, is not determined but only a reflected andsubsequently processed image of the environment is projected back intoan eye on the same beam path in order to ensure a superimposing with theactually perceived image.

In the still unpublished Application PCT/EP01/05886, many differentmethods and devices are also described for the adaptation of an opticalsystem to the direction of vision of the human eye.

It is therefore an object of the present invention to provide a deviceand a method by means of which the position and/or the orientation,particularly the direction of vision, of an eye can be determinedrapidly and precisely.

SUMMARY OF THE INVENTION

This object is achieved by the characteristics claimed. Preferredembodiments of the invention are also claimed.

In the case of a device according to the invention or a method accordingto the invention for determining the position and/or the orientation,particularly the direction of vision, of an eye, a starting point or anend point of a light beam reflected by a part of the eye and detected bya detector system and/or of a light beam projected by a projectionsystem onto or into the eye quasi two-dimensionally describes a movementpattern of a scanning and/or projection movement in the eye when thedirection of the light beam is changed with respect to the timeaccording to the scanning or projection movement.

A device according to the invention or a method according to theinvention for determining the position and/or the orientation,particularly the direction of vision, of an eye according to the presentinvention is therefore based on the fact that that a light signalreflected by a part of the eye is detected by means of a detector systemand is analyzed. In addition to the detector system, a device accordingto the invention may comprise additional optical devices, particularly aprojection system.

The term “position” or “orientation” may apply to the position ororientation of the eye relative to the device (relative position) orrelative to the environment (absolute position). In the following, forreasons of simplicity, position and/or orientation are sometimescombined in the term “orientation”. The term “orientation” may thereforealways apply to the kinematic orientation, to the kinematic position orto both.

As required, in the knowledge of the position or orientation of thedevice relative to the environment, which is determined, for example, bymeans of a suitable orientation determining device (laser triangulation,radar, Global Positioning System (GPS) receiver, and the like, bothpositions can be unambiguously converted to one another according to therules of kinematics. In this case, the orientation determining devicemay be fastened to the device according to the invention itself (such asa GPS receiver with a pertaining analyzing unit), and/or can determinethe orientation of the device according to the invention from theenvironment (for example, by means of laser triangulation, or the like).The determination of the orientation of an eye may also comprise thedetermination of a change of orientation of the eye with respect to aselected reference position.

In a preferred embodiment, the device according to the invention may beconstructed in the form of spectacles. Likewise, it may, for example,also be arranged on or integrated in a frame wearable on the head, ahelmet, or the like. In the same manner, it may, for example, also bearranged on or integrated in a stationary apparatus which uses theinformation concerning the orientation of the eye supplied by the deviceaccording to the invention, for example, in an apparatus for lasersurgery or the like. Furthermore, it may be arranged on or integrated inan object which is movable relative to the environment and relative tothe carrier, as, for example, a portable electronic notebook, a laptopor the like.

In the following, various characteristics are now suggested for each ofthe terms “part of the eye”, “light signal”, “optical device”,“detection and analysis”, which, in the case of a device according tothe invention or a method according to the invention may be combined asan alternative or jointly.

It is pointed out expressis verbis that the present invention comprisesall conceivable combinations of the described characteristics, unlessthe combination does not make any sense to the person skilled in the artfrom the start.

Wherever it is expedient, characteristics of a method according to theinvention will be described in the following. In this case, a device isalso always explicitly disclosed which is suitable for implementing thesuggested method; for example, an appropriately programmed computer;sensors which are capable of supplying the necessary informationsignals; signal processing devices which are capable of appropriatelyprocessing these signals (filtering, digital-to-analog oranalog-to-digital converting, storing, or the like) etc. Inversely, acorresponding method is also always disclosed by means of the functionof the device according to the invention.

In the following, preferably the straight line through the pupillarycenter and the fovea centralis is defined as the direction of vision ofan eye because, as a rule, a human being will direct his eye to a pointto be viewed in such a manner that this point is imaged in the region ofthe sharpest vision. Bibliographically, a multitude of different axes ofthe eye is used, such as the fixation axis through the fixed viewedpoint and the fovea centralis, the visual axis through the nodal pointand the fovea centralis or the achromatic axis—the axis with adisappearing transversal chromatic aberration—, which are each definedby geometrical relationships (straight lines through the foveacentralis, the pupillary center, the lens center, the focal point,etc.). The present invention is not limited to the above-mentioneddefinition of the direction of vision. As an alternative, any of thestraight lines familiar to a person skilled in the art can be used as adirection of vision.

1. Light Signal 1.1 Definition

The essence of the invention is a light beam reflected by a part of theeye, which light beam is used for determining the orientation andparticularly the direction of vision of the eye. In this case, a beam ofrays is always called a light beam in accordance with geometricaloptics. Its diameter may be very small; in borderline cases, infinitelysmall (the light beam will then change into the so-called principalray), preferably less than or equal to the entrance pupil of thedetector. In this case, it was surprisingly found that, contrary to aprejudice existing in the state of the art, also very small beamdiameters can be used when sufficiently strong light sources (such aslasers or the like) and/or sufficiently sensitive detectors are applied.Furthermore, it is explicitly pointed out that the term “light beam” maycomprise electromagnetic radiation of any wavelength, in addition to thevisible, thus particularly also infrared light. The term “reflected”light beam comprises light beams generated in the eye, for example, thethermal radiation of blood vessels in the retina as well as light beamsreflected or scattered on parts of the eye, which fall upon the eye fromthe outside.

1.2 Passive Scanning

Ambient light can preferably be used as reflected light, which ambientlight which is incident in the opened eye from the environment and isreflected there on at least one part, such as the retina. Subsequently,in a pixel-type manner, one light beam respectively is selected anddetected or intercepted from this reflected light in a targeted mannerby a detector system. In the following, this is called “passivescanning”.

Reflected light may also be light which is emitted by a part of the eye,for example, thermal radiation. Subsequently, in a pixel-type manner,one light beam respectively is selected and intercepted or detected fromthis emitted light by suitable devices. In the following, this is alsocalled “passive scanning”.

Advantageously, no projection system is required here. Neither is aninterfering additional image signal projected into the eye. Furthermore,light beams of particularly suitable wavelengths can be filtered out ofthe polyspectral ambient light.

1.3 Active Scanning

As an alternative, according to the invention, light can first beactively projected into the eye by means of a projection system, such asa laser or another radiator, and a beam reflected at the correspondingpart of the eye can subsequently be intercepted or detected. In thefollowing, this is called “active scanning”.

1.3.1 Modulation

According to the invention, the light actively beamed into the eye maybe appropriately modulated, for example, in order to be able to easilydifferentiate it from the ambient light by means of a correspondingsignal processing or to be able to determine the point in time of itsemission. In this case, one or more characteristic quantities of theactively beamed-in light, such as the intensity, the wavelength orfrequency, the polarization, the radiance or the like, are alwayschanged over time.

1.3.1.1 Amplitude Modulation

In the following, amplitude modulation is a modulation in which at leastone of the above-mentioned characteristic quantities each have definedvalues in different time segments or at different points in time. Alight that is modulated in such a manner can be differentiable, forexample, from ambient light on the basis of the characteristic quantityand/or the point in time of its emission can be determinable.

1.3.1.2 Frequency Modulation

As an alternative to or together with the above-described amplitudemodulation, the actively beamed-in light signal may also befrequency-modulated. In the following, frequency modulation is thechange of a periodicity of at least one of the above-mentionedcharacteristic quantities; i.e., one of the quantities changesperiodically, and the period length of this periodic change has definedvalues in different time segments or at different point in time. Bymeans of the period length of a reflected detected light signal, forexample, the point in time of the emission of this light signal andthereby the propagation time of the light signal can be determined.

1.3.1.3 Other Modulations

One or more characteristic quantities of the actively beamed in lightbeam may also be simultaneously or successively amplitude- and/orfrequency-modulated. Furthermore, analogously, not only the value(amplitude modulation) of at least one characteristic quantity of thelight beam and/or the periodicity (frequency modulation) of thisquantity can be modulated, thus, have respectively defined values indifferent time periods, but, for example, also the phase angle ofperiodically changing quantities (phase modulation).

The light signal may also be modulated in a different manner such thatit contains information, for example, as to when it was emitted orinformation characterizing the light signal in contrast to ambientlight.

1.3.2 Adaptation of the Active Luminous Intensity

In contrast to the passive scanning, the active scanning has theadvantage that it can be ensured that light of sufficient intensity isalways reflected from the eye in that a correspondingly large amount oflight is actively beamed into the eye and reflected by it.

1.3.3 Beam Diameter

1.3.3.1 Point-Focal Illumination

According to the invention, a light beam, which may have a very smalldiameter, can be emitted by a radiator, such as a laser, and can bedirected such that successively, i.e., sequentially, different pointscan be illuminated on the part of the eye (point-focal illumination).

The light beams reflected at these points can be detected in a targetedmanner by a detector system (point-focal scanning). For example, a lightbeam is emitted in each case and the reflected light beam is detected inthe detector system.

Likewise, in the case of a point-focal illumination, a detector systemcan also detect a larger area; i.e., it can simultaneously detect lightbeams which are reflected at difference points (areal scanning). Since,in each case, only one point is actively illuminated, the detectorsystem in each case also only receives one actively beamed-in andreflected light beam. This is advantageous because the detector systemdoes not have to be successively focused on different points or beadjusted to light beams of different points. It should be noted that apoint-focal or areal scanning is also conceivable in the case of apassive scanning and/or areal illumination.

1.3.3.1.1 Detection of Light from the Entire Pupil

A light beam beamed into the eye, which preferably may have a very smalldiameter, at a reflecting part of the eye, for example, the retina, isfor the most part reflected back or scattered back into a defineddirection. A portion of these light beams reflected or scattered intoanother than the defined direction leaves the eye through the pupil andextends approximately coaxial with respect to the reflected principalray. The reflected light beams which just barely still pass through thepupil are called marginal rays. The detector system is preferablyconstructed such that these marginal rays can also be detected, wherebythe detectable light quantity is advantageously increased.

1.3.3.2 Areal Illumination

Instead of an individual light beam having a possibly very smalldiameter, a larger area of the eye can also be actively illuminated in aareal manner (areal illumination). For example, a light beam having alarger diameter can be directed onto the eye such that in each case anentire area of the retina or of the cornea is illuminated and reflectslight. From this reflected light, a reflected light beam used fordetermining the orientation can then be analogously selected for thepassive scanning by the detector system (point-focal scanning).

As in the above-described areal scanning, the detector system can alsodetect a larger area.

The actively beamed-in light can appropriately be modulated such that itcan be selected particularly well in that it has, for example, a definedwavelength, polarization, etc. The illuminated area may comprise theentire part of the eye on which light is reflected and detected for thedetermination of the orientation. As an alternative, partial areas mayalso be illuminated successively so that in each case at least the pointat which light is reflected and detected for determining the orientationof the eye is illuminated.

1.4 Mixed Operation

According to the invention, active scanning and passive scanning may becombined. For example, the light perceived by the eye is detected fromthe reflected ambient light and additionally a light beam is activelybeamed in order to detect certain structures, marks or characteristicsof the eye (fovea centralis, blind spot, etc.). The combination mayadvantageously be variable so that, for example, in phases of sufficientambient light intensity, no active scanning takes place and, as soon astoo little ambient light is reflected, additional light is activelybeamed in.

Advantageously, different passive and/or active scanning methods canalso be combined. Thus, the advantages of both methods can be combined.For example, actively beamed-in light that is reflected on the retinacan be detected in the infrared range or infrared light emitted by theblood vessels of the retina can be detected; i.e., many light beams canbe detected in order to, for example, precisely detect individual bloodvessels for determining the orientation. Simultaneously, an image of theenvironment reflected by the retina can be detected in order todetermine a correlation of the perceived image with the blood vessels.From the correlation between the blood vessels fixed with respect to theeye and an image fixed with respect to the environment, the orientationof the eye relative to the environment can then be determined.

1.5 Wavelength

In the case of the passive scanning, as described above, certainwavelengths can be filtered, i.e., absorbed or permitted to pass fromthe ambient light, which may comprise the range of the visible light aswell as the remaining spectrum of electromagnetic radiation,particularly infrared radiation, which wavelengths can, for example, bedetected particularly well. In the case of the active scanning, alsovisible light as well as electromagnetic radiation of other wavelengthranges, particularly light from the ultraviolet and infrared range, canbe used. Infrared light is particularly advantageous because theadditional active signal is not perceived by the eye.

1.6 “Flying Spot”

In the case of the passive as well as in the case of the activescanning, reflected light beams are preferably serially intercepted andanalyzed; i.e., light beams which are reflected by certain points(“pixels”) along a curve on the concerned part of the eye aresuccessively detected by the detector system. This sequential scanningis called a “flying-spot” method because here the point of vision(“spot”) of the detector system quasi moves along the curve. Generally,scanning and projection methods by which spatially bounded areas aretypically serially scanned or illuminated along a straight or bent curveare frequently called “flying-spot” methods in technical terminology.

In the present application, mathematically any arbitrary continuous ordiscrete sequence of two-dimensional or three-dimensional points iscalled a curve, in which case, one or more values may be assigned toeach point. In order to avoid confusion, the curve of the points of theeye which are illuminated or scanned will in the following be called atrajectory or pattern of movement. This means that the starting or endpoints of the illuminated or detected light beams describe a two- orthree-dimensional pattern of movement in the or on the eye. The patternof movement which is the result of the scanning or detection of thelight beams is called the pattern of movement of the scanning movementor scanning pattern. The pattern of movement which is obtained duringthe projection of the light beams is called the pattern of movement ofthe projection movement or the projection pattern.

In the case of the areal scanning, a three-dimensional, as a rule,curved structure of the eye is imaged on a two dimensional picture.Undesirable distortions occur in this case. Furthermore, the focusingonto a three-dimensional surface is problematic.

In the case of many conventional eye-related information systems,optical signals are detected by means of a flat detector flatly from theeye or are projected by means of a flat projector flatly into the eye.This approach has the disadvantage that an optically correct imaging ofa curved eye part on a flat detector or of a flat projector onto acurved eye part can be achieved only at considerable expenditures. Thisproblems occurs to a considerably reduced extent in the case of theflying-spot method.

Advantageously, in the case of the sequential or point-focal scanning,one point respectively of a structure of the eye is imaged on a scanningelement. As a result, the above-described distortion or focusingproblems can be reduced.

The use of light beams of a small diameter is therefore alsoadvantageous. The used light beams at the air/eyeball transitiontherefore advantageously have a diameter of less than 100 μm, preferablyof less than 50 μm, particularly preferably of less than 10 μm, orparticularly advantageously of less than 5 μm.

In addition, because of its compatibility with a holographic element,the flying-spot method is preferably used in the method according to theinvention.

1.6.1 Spiral, Circular or Elliptical Scan

When mechanically controlled deflection devices are used, such asmirrors or the like, circles, ellipses or spirals are advantageouslyused as curves because, in this case, the deflection devices carry outharmonic movements (for example, a sinusoidal up-down movement or thelike; see Section “Optical Devices”), which considerably reduces theexcitation of undesirable oscillations of these devices. This ensures ahigher precision of the determination of the orientation of the eyebecause of reduced systematic errors during the beam deflection.Likewise, for example, a lattice line pattern can be followed.

Even without mechanical deflection devices, for example, circles,ellipses or spirals are advantageously used as curves: The humanmonocular perception essentially is rotationally symmetrical about avisual axis extending through the fovea centralis and the optical centerof the lens. Correspondingly, many parts of the eye, such as the iris,the pupil, the cornea, the lens and, in some aspects, also the retina inmost people are constructed approximately rotationally symmetricallyabout the visual axis.

According to the invention, preferably according to the flying-spotmethod, the eye is therefore scanned by means of a spiral- orcircle-shaped scanning or projection pattern, preferably about thevisual axis, in which case, “circle-shaped” may mean a plurality ofconcentric circles. When the projection or scanning beams are disposedcorrespondingly diagonally with respect to the visual axis, the use ofan elliptical scanning or projection pattern may be advantageous, asdescribed in German Patent Document DE 197 28 890 A1. To this extent,reference is made to the full content of German Patent Document DE 19728 890 A1.

1.6.2 Beams Perpendicular with Respect to the Eye

When light beams are perpendicularly incident on the air/eyeballtransition, a certain fraction of the light is reflected back in theopposite direction to the incident light beam, while the remainingfraction penetrates quasi unhindered, after which it is absorbed orscattered by “lower-lying” fractions of the eye. The former analogouslyapplies to light beams exiting from the eye by way of the cornea/airtransition.

In the case of the method according to the invention, the projection orscanning preferably takes place according to the flying-spot method. Inthis case, a “narrow” light beam is preferably used which, at theair/eyeball transition, has a diameter that is insignificant incomparison to the curvature of the eyeball, particularly with respect tothe curvature of the cornea. The light beam is preferably projected orscanned such that all its individual rays meet the air/eyeballtransition in a manner as perpendicular as possible.

The cornea, i.e., the air/cornea transition, causes approximately 80% ofthe refraction exercised by the eye upon an incident light beam. Thus,the above-described approach not only has the advantage that littlelight is refracted at the air/cornea transition into a uselessdirection, but also has the advantage that the beams experience a slightrefraction by the optical system of the eye. This has a positive effectnot only on the spatial projecting or scanning precision but is alsoadvantageous for applications in which the geometry of the light beamsplays a significant role.

The fact that beams incident perpendicular to the eye are partiallyreflected back in the opposite direction can be used for obtaininginformation concerning the topology of the eye. This can take place, forexample, by way of a projector—detector arrangement comprising aprojection system and a detector system, which arrangement projectslight approximately perpendicularly onto the eye and subsequentlydetermines the coaxiality of the detected reflected-back light beam andof the projected light beam. When these light beams are not essentiallycoaxial (particularly the surface of the cornea has many microscopic andmacroscopic irregularities and should therefore not be considered as asmoothly reflecting surface), the conclusion can be drawn that theprojected light beam was not perpendicularly incident on the eye. Suchinformation concerning the topology of the eye can be used for, amongother things, the determination of the position and/or orientation ofthe eye.

A confocal arrangement of the projection system and of the detectorsystem, for example, by way of splitter mirrors, is useful for such aprojector-detector arrangement.

The use of a holographic element is advantageous particularly for theprojection or the scanning of a light beam extending perpendicular withrespect to the eye, because such a simple virtual development of a lightguiding device makes it possible that light beams can be directed from asingle projection system perpendicularly onto various regions of the eyeand/or light beams perpendicularly exiting or reflected-back fromvarious regions of the eye can be directed into a single detectorsystem.

The various light signals which can be used for a device according tothe invention as well as for a method according to the invention weredescribed above. Next, the part of the eye will be discussed from whichthese light beams can be reflected.

2. Reflecting Part of the Eye

For determining the orientation and particularly the direction of visionof the eye, a light beam is detected which is reflected on at least onepart of the eye. As a result of their reflection or emission behaviorand particularly because of certain characteristic features existing onthem, various areas of the eye can be used which will be discussed indetail in the following. However, light for determining the orientationof the eye can also be used that is reflected by other parts of the eyenot explicitly mentioned here.

2.1 Cornea

According to the invention, light beams can be used for determining theorientation of the eye which are reflected by the cornea. Preferablythose light beams are used in this case that are reflected on theexterior surface which is in the front in the direction of vision.

On its exterior surface, the cornea always has a plurality ofmicroscopic (for example, small scars caused by foreign particles, suchas dust, etc.) and/or macroscopic (for example, as a result of asurgical intervention on the cornea) irregularities which, although theyhave partially disappeared after a few days because of healingprocesses, for a sufficient time period, each represent a characteristicmark which is fixed to the body and can be optically detected withrespect to the orientation of the eye. Likewise, the cornea may alsohave other significant characteristics, such as cloudiness ordiscolorations, which can also be used for determining the orientation.

For example, for determining the orientation of the eye, one or more ofthese irregularities can be detected and this irregularity orirregularities can be “followed” over time; i.e., for example, theirrespective position can be detected in a two-dimensional image. A changeof the orientation of the eye can then be determined from the change ofposition of the irregularity or irregularities.

Likewise, for example, for determining the orientation of the eye, areference map of the cornea can be established on which the position ofdifferent significant characteristics is stored with respect to areference system of coordinates. When the eye is now scanned again inthe same manner, an image or pattern identification can recognize thesignificant characteristics in the newly established map and, bycomparing the position and orientation of the characteristics on thereference map or new map, can determine a change of the orientation ofthe eye (see also the “Analysis” section).

As an alternative or in addition, a light beam emitted or reflected byor at the surface of the cornea which is in the rear in the direction ofvision can also be detected. By means of this light beam, for example,the characteristically dark reflex image of the macula fovea on the rearsurface can be detected whose position on the cornea points directly tothe orientation of the eye. Likewise, the phase shift between the lightbeams reflected on the frontal and on the rear surface can be determined(for example, according to Purkinje).

2.2 Sclera, Iris, Pupil

According to the invention, light beams can be used which are reflectedby the iris or the adjoining sclera. Because of the considerablydifferent degrees of reflection of the white sclera, the colored irisand the quasi light-swallowing pupil, as a rule, the transitionssclera/iris and/or iris pupil can be determined very easily andaccurately from the reflected beams. As a result, for example, theposition of the pupil and/or of the iris can be determined in that, bymeans of a considerably changing brightness of the reflected light aconclusion can be drawn with respect to the iris/pupil transition andthus points can be found on the circumference of the pupil (see“Analysis” section).

In addition or as an alternative, the structure of the iris can also bedetermined. The latter has characteristic features (patterns, colordifferences, injuries or the like) whose position can, for example, bestored in a reference map and can be compared with the actuallyidentified position of the same features in order to determine anorientation of the eye. As in the case of the cornea, an orientation ofthe eye can therefore be determined by means of the detection of certaincharacteristic features.

2.3 Retina

According to the invention, light beams can preferably be used which arereflected by the retina. Characteristic features of the retina can bedetermined therefrom, such as individual larger blood vessels or themacula with the fovea centralis, which permit not only the determinationof the orientation but, for example, together with the pupillary center,particularly also the direction of vision of the eye, which can bedefined as a straight line through the pupillary center and the foveacentralis (see also the “Analysis” section).

Likewise, it is possible to detect an environment reflex image from theambient light reflected by the retina, which allows conclusions on theperceived environment. For example, the orientation change of the eyerelative to the environment can then be determined from a time-relateddisplacement or distortion of the environment reflex image.

For the purpose of differentiation, in the following, the image of theenvironment, which is generated by light incident on the retina from theenvironment and at least partially reflected by the retina, is called anenvironment reflex image (of the retina) (passive scanning, while theimage which is the result of light actively beamed into the retina andreflected by the latter is called a retina reflex image (activescanning). In addition, the term “retina reflex image”, for the purposeof a more compact representation of the different variants, alsocomprises an image emitted by a part of the eye itself, such as theimage of the heat radiation which the retina emits although thisactually is not reflected light.

After the explanation of the different light signals and of differentparts of the eye from which these can be reflected and used fordetermining the orientation of the eye, optical devices will bediscussed in the following by means of which the light beams can beintercepted and, as required, reflected beforehand.

3. Optical Devices

The present invention comprises a detector system. It comprises adetector for detecting light beams which, as described above, arereflected by a certain part of the eye. In addition, the detector systemmay comprise a first light guiding arrangement (detector light guidingarrangement), for the deflection and/or focusing of light beams by meansof which, for example, successively different light beams reflected by apart of the eye can be guided into the detector and/or can be focused onits receiver area. This guiding device is preferably designed such thatlight beams, which were reflected from different points of the eye, aresuccessively, i.e., sequentially guided into the detector, so that thelatter detects a corresponding sequence of scanning elements.

For the purpose of an active scanning, a device according to theinvention, in addition to the detector system, may comprise at least oneadditional optical device, specifically a projection system forprojecting light. The projection system comprises a radiator for theemission of light. Furthermore, the projection system may comprise alight modulation device in order to, for example, appropriately modulatethe light emitted by the radiator. In addition, the projection systemmay comprise a projection light guiding arrangement, for example, forthe deflection and/or focusing of the emitted light beams, by means ofwhich, for example, successively, light beams, which are emitted by theradiator, are guided and/or focused onto certain points of the eye.Preferably, the detector light guiding device is at least partially usedby the projection system. For example, light can first be activelybeamed onto the eye and, on the same beam path, the reflected light canbe guided into the detector. As a result, at least partially the samebeam path is used for the projection and the detection. This isadvantageous because, since systematic defects of the light guidingdevice can be compensated, the projection system can at least partiallydo without its own light guiding arrangement. Furthermore, it isadvantageous that the detector quasi automatically detects the lightsignal emitted by the radiator and does not especially have to beadjusted to the reflecting point of the eye. In the following, theseindividual elements of optical devices according to the invention willbe explained in detail.

3.1 Detector

A detector system according to the invention comprises a detector fordetecting light beams. The detector has a receiver area and detects atleast one characteristic quantity of light beams which impinge on thisreceiver area, such as the radiation energy or the radiated power, theirradiance, the wavelength or frequency, the polarization or the like.The detector emits a signal corresponding to the detected quantity orquantities. By means of a suitable device, such as an analog-to-digitalconverter and/or a filter, as necessary, this signal can beappropriately processed before the emission. For example, anopto-electronic detector can detect an optical signal and emit acorresponding electrical signal; an opto-optical detector can detect anoptical signal and emit a corresponding optical signal.

The detector may, for example, comprise a photodiode, in which currentflows during the irradiation, from whose intensity the illuminance canbe determined. The detector may also have a photo resistor whose ohmicresistance changes as a function of the absorbed light. From the changeof the resistance, in turn, the irradiation can be determined. Thedetector may also comprise a photocell, a phototransistor, aphotomultiplier, an image intensifier tube or basically any othercomponent part that permits the detection of a physical quantity of alight beam impinging on the component part.

3.2 Detector Light-Guiding Arrangement

The above-described detector preferably sequentially detects light beamswhich are reflected by different points of the eye. For this purpose,the detector itself may be correspondingly constructed or controlledsuch that it can sequentially detect light beams in a targeted mannerwhich impinge from different directions or on different points of itsreceiver area. As an alternative or in addition, a detectorlight-guiding arrangement may be present which in each case guides alight beam into the receiver area of the detector and/or focuses thelight beam onto the receiver area. The detector light-guidingarrangement can be controlled such that light beams are sequentiallyguided onto the detector which impinge on the detector light-guidingarrangement from different directions. By means of a correspondingtime-related controlling of the detector light-guiding arrangement, thelight beams can be directed in a targeted manner sequentially onto thedetector which were reflected by certain points of the eye. The spot ofthe detector thereby describes a pattern of movement on a part of theeye; it therefore scans this area by means of a certain pattern ofmovement.

Any device can be used as the detector light-guiding arrangement thatfulfills this function; thus, can be controlled such that light beamsare sequentially guided and/or focused from different, defined ordefinable direction in each case onto the receiver area of the detector.In the following, several conceivable characteristics of such deviceswill be explained in detail.

3.2.1 Mirrors

A detector light-guiding arrangement according to the invention maycomprise one or more mirrors. Their orientation to one another, to theeye, whose orientation is to be determined, and to the detector can beadjustable such that light beams reflected by certain points of the eyeare guided to the detector. Some of these mirrors may be arranged in afixed manner with respect to the detector (“fixed mirrors”), while othermirrors may be adjustable in their orientation (“movable mirrors”).

The orientation of the movable mirrors can be controllable such that adesired pattern of movement can be scanned; i.e., that light beams whichwere reflected by certain points of the eye are sequentially guided ontothe detector. Because of the reversibility of the beam path, thedetector light-guiding device can also be understood as if an imaginedlight beam were emitted by the detector and were guided and/or focusedby the detector light-guiding arrangement onto the point of the eyewhich specifically is to be scanned by the detector.

3.2.2 Holography

Instead of the above-described mirrors, holographic elements can also beused which cause the desired deflection of the light beams. Comparedwith conventional mirrors, such holographic elements have severaladvantages. On the one hand, they have a lower weight than a mirror. Onthe other hand, they can be further developed such that they are notperceived by the eye itself. It is mainly possible to produceholographic elements having almost any reflection behavior. For example,a holographic element may reflect light only in a certain wavelengthrange. When this wavelength range is, for example, in the infraredrange, the holographic element cannot be perceived by the eye.

A holographic element may, for example, be a holographic coating on acarrier.

When, instead of a movable mirror, a holographic element is used for thechangeable deflection of light beams, the change of the orientation ofthe reflection required in this case can take place analogous to themovable mirror by an orientation change of the holographic element.However, the reflection behavior of the holographic element can, forexample, also be changed electronically, which permits a considerablyfaster and more precise scanning. For this purpose, the holographicelement has to be electro-holographic.

3.2.3 Other Elements

The detector light-guiding device may comprise other optical elements,particularly, for example, apertures, which limit the diameter of thedetected light beam and thereby spatially limit the scanned area of thescanned part of the eye, or optical or electro-optical lenses whichappropriately expand or focus the light beam, for example, on thereceiver area of the detector.

3.3 Projection System

In addition to the detector and possibly the detector light-guidingarrangement, a projection system may be present for an active scanning,which projection system has a radiator that emits light beams. Further,the projection system may comprise a light modulation device forappropriately modulating the light emitted by the radiator. In addition,the radiator may comprise a projection light-guiding arrangement, forexample, for the deflection and/or focusing of the emitted light beams,by means of which light beams emitted by the radiator are, for example,sequentially guided to and/or focused on certain points of the eye.These elements will be explained in the following.

3.3.1 Radiator

A radiator emits light beams which preferably impinge on a part of theeye and are reflected by the latter. In principle, any device that canemit light beams is therefore conceivable as a radiator, such aselectric bulbs, lasers, LEDs or the like.

Advantageously, at least one of the physical quantities emitted by theradiator is adjustable, so that the emitted light can be differentiatedfrom the ambient light. For example, a laser may only emit light in acertain narrowly restricted wavelength range, or the radiator emitslight beams having a certain time-related pattern with respect tointensity, wavelength or the like.

In the case of the present invention, the term “radiator” may alsocomprise several individual light sources, such as varicolored LEDs orlasers which can emit light of different wavelengths.

3.3.2 Light Modulation Device

The light emitted by the radiator can be appropriately modulated in alight modulation device before it impinges on the eye. For example, alight modulation device may comprise a color filter which allows onlylight of a certain wavelength to pass. Likewise, a light modulationdevice may comprise a polarization filter which allows only light of acertain polarization to pass. Such and other filters may be controllablein such a manner that the light can be modulated over the time.

3.3.3 Projection Light-Guiding Arrangement

In order to guide light emitted by the radiator to from where it is tobe reflected in order to subsequently be detected by the deviceaccording to the invention, a projection light-guiding arrangement maybe present into which light enters that was emitted by the radiator andwhich guides this light to the desired area.

In principle, because of the reversibility of the beam path, anarrangement can be used for this purpose that is analogous to any of thedescribed detector light-guiding arrangements.

In particular, the detector light-guiding device itself may form a partof the projection light-guiding arrangement in that the light emitted bythe radiator enters in the opposite direction parallel into the beampath of the light detected by the detector system in front of, in orbehind the detector light-guiding device. Such an arrangement has theadvantage that possibly existing systematic defects are identical duringthe projection and scanning and compensate one another. Anotheradvantage consists of the fact that the detector system quasiautomatically detects the light beam which the radiator has emitted.

For this purpose, a splitter mirror may, for example, be present betweenthe detector light-guiding device and the detector, which partiallyallows light coming from a radiator to pass into the detectorlight-guiding device, and partially reflects light coming from detectorlight-guiding device to the detector, in which case beams in thedirection of the detector are preferably given priority. For example,the ratio of the beams reflected to the detector to the beams incidenton the splitter mirror may amount to 95%, 90%, 85% or 80%. The ratio ofthe beams passing through the splitter mirror to the beams incident onthe splitter mirror may, for example, amount to 5%, 10%, 15% or 20%.This means that a light beam impinging on the splitter mirror isreflected, for example, by 95% and can pass through the mirror by 5%.

The fact that such a splitter mirror hinders the projection beam isuncritical because this can be compensated by an increase of theradiator power.

3.4 Exterior Picture

In addition, a device according to the invention may comprise a devicefor taking a picture of the environment, such as a camera. This picturecan be processed in order to, for example, identify significant patternsof the environment.

A picture of the environment taken by such a device can be compared in asuitable manner, for example, with an image of the environment(“environment reflex image”) reflected at the retina or the cornea ofthe eye. For example, in both images, the same object can be identifiedand, from the spatial assignment of the two images of the object, theorientation of the device or of the eye relative to the environment canbe determined.

Such a device for taking a picture of the environment can preferably bearranged approximately confocal with respect to the eye.

3.5 Position of the Device

In addition, a device according to the invention may comprise anorientation determining device for determining the orientation of thedevice in order to determine, for example, by means of an orientation ofthe eye relative to the device, an orientation of the eye relative tothe environment.

Such an orientation determining device may be fixedly connected with thedevice according to the invention and determine the orientation and/orposition of the device relative to the environment. An orientationdetermining device may, for example, comprise a GPS receiver and thepertaining analysis device which determines the position of the devicefrom the received GPS signals.

Likewise, an orientation determining device may also be fixedlyconnected with the environment and determine the orientation of thedevice, for example, by means of triangulation or the like.

3.6 Markers

A device according to the invention may contain one or more markers. Amarker may, for example, be arranged in front of the eye, either insideor outside the field of vision. Such a marker may, for example, bearranged on a spectacle lens in front of the eye.

A marker may, for example, be used for determining optical referencevalues. For example, a light beam reflected on such a marker may bedetected by the detector. A characteristic quantity of this light beamcan then be determined as a reference value, for example, for a 100%reflection.

Such a marker may also be used as a fixed point during the determinationof the orientation of the eye with respect to the device. For example,such a marker may be present in such a manner that, in the addition tolight beams reflected by a significant region of the eye (for example,the fovea centralis or the iris), the detector can also detect lightbeams which are emitted by the marker, for example, reflected. By meansof the directions from which the respective light beams are detected, aposition of the significant region of the eye relative to the marker andthus an orientation of the eye relative to the device can be determined.

Such a marker may also be used for calibrating or recalibrating theoptical device. As a result of outside influences (such as a shock) orinside influences (such as a temperature (caused?) elongation), in thecourse of the operation, the position and orientation of individualelements of the optical device may change. For example, because of adeformation of the device on which they are arranged, the mirrors maychange their position with respect to one another. In order to determinesuch a change, for example, a reference position of the marker can bedetermined first in that it is determined how the detector light-guidingarrangement has to be controlled so that the detector will detect lightthat is reflected by the marker. When then later, the position of themarker is determined analogously, the change of the optical device withrespect to the marker can be determined from a change with respect tothe reference position. When the marker is fixed with respect to thedetector, the (apparent) change of position is based on a change of thedetector light-guiding arrangement which can thereby be determined.

During the active scanning, a light beam can advantageously be beamed atthe marker in a targeted manner and the reflected light beam can bedetected by the detector, for example, in order to make one of theabove-mentioned determinations (reference value, fixed point,calibration).

The marker may advantageously be further developed so that it cannot beperceived by the eye. The marker may, for example, only be visible inthe infrared range, so that it can be detected by an infrared detectorbut is not perceived as being disturbing by the human eye. This can beimplemented, for example, by means of a hologram.

As an alternative or in addition, a marker itself may also actively emitlight.

Additional markers according to the invention or their use are disclosedin Section 4.3.

3.7 Focusing

The above-described optical devices, thus particularly the detectorsystem and/or the projection system may comprise at least one suitablefocusing device by means of which, for example, the distance of thescanned area from the detector and/or radiator can be adjusted, and itcan thereby, for example, be determined whether the retina or the corneais scanned or illuminated. In the case of light beams having smalldiameters, the benefit of such a focusing device may be limited.

After the listing and detailed explanation of the characteristics of theoptical device which detect a light beam reflected by a part of an eyeor beam light onto the eye, the detection of the light beams and theanalysis of the thus obtained signals will be discussed in thefollowing.

4. Detection and Analysis

It is an object of the present invention to determine the positionand/or orientation, particularly the direction of vision, of an eye. Inthe following, it will be described how characteristics of the eye canbe determined from light beams reflected by a part of the eye andsubsequently detected by the detector system, and how thesecharacteristics can be used for determining the orientation of the eye.

4.1 Detected Image

During the determination according to the invention, points arepreferably scanned sequentially, i.e., after one another, along atrajectory. This means that light beams which are reflected by thesepoints are sequentially detected in the detector system and at least onephysical quantity of these light beams is determined, such as theirintensity or brightness, radiation output, wavelength, gray-scale valuesor the like. The result is, for example, a sequence of scanning elements(i.e., a scanning element curve) to which, for example, values of thephysical quantity or quantities and, for example, coordinates determinedby means of the position of the detector light-guiding arrangement canbe assigned (for example, abscissa x and ordinate y; radius R and polarangle Φ). The values of the physical quantity or quantities are alsocalled an information content of the light (signal).

In the case of a finer scanning, i.e., for example, in the case ofsmaller steps between the individual positions (x,y) of the detectorlight-guiding arrangement, an increasingly sharper two-dimensional“picture” of the part of the eye can be determined from the pictureelement curve, by which the light beams are reflected. By means of thescanning according to the invention, a two-dimensional picture of atleast one three-dimensional region of the eye can therefore beestablished, for example, of the cornea, the retina or the like.

If one of these regions is curved, it appears distorted in the twodimensional image. The orientation of the eye can be determined fromsuch distortions. Depending on the orientation of the eye with respectto the detector, for example, a circular pupil may appear as an ellipsefrom whose principal axes the orientation of the eye can be determined.

Elements situated behind one another in the direction of view of thedetector or of the detector light-guiding arrangement, i.e., against thedirection of the light beams which are detected by the detector,mutually overlap in the two-dimensional image or have a certaindistance. On the basis of such overlaps or distances, the orientation ofthe eye can in turn be determined. When, for example, thethree-dimensional position of the pupillary center with respect to thefovea centralis is known, for example, empirically from statisticalexaminations or preceding determinations according to the invention,from the spacing of the images of the pupillary center and of the foveacentralis in the two dimensional image, their spatial position andthereby the orientation and/or position of the eye can be determined.

Likewise, the three-dimensional position of a detected point can bedetermined, for example, from the two-dimensional position of itspicture element and the propagating time of the signal reflected on it,in which case the propagating time can be determined, for example, inthe case of a frequency-modulated light signal by means of the frequencyof the detected light signal. From the propagating time, for example, adistance of the detected point from the detector and thus the positionof this point relative to the device can be determined. Likewise,instead of the propagating time, the focusing, i.e., for example, theadjusting of a focusing device can be used, which also suppliesinformation concerning the distance of the detected point from thedetector.

When, for example, by means of the propagation time, the distance of thereflecting point from the detector is determined, a z-value canadditionally be assigned to each picture element, so that thethree-dimensional position of the scanned points relative to the deviceaccording to the invention can be determined. In this case, a quasithree-dimensional image of the eye can be scanned. For reasons ofsimplicity, in the following, reference is mostly made totwo-dimensional images. However, by means of the above-illustratedthree-dimensional picture elements, an analogous three-dimensionalmethod may also always be used. When, for example, a position of thepicture element in the two-dimensional image is mentioned, by means ofan additional measuring of the distance of the point, which reflectslight, with respect to the detector, the three-dimensional position ofthe point relative to the device or relative to other points can also bedetermined.

The two-dimensional images of the same three-dimensional structure ofthe eye, which is scanned in various orientations, are not only shiftedor rotated with respect to one another (from which, in reverse, theorientation change of the eye can be determined) but are also distorteddifferently. At a preferred high scanning frequency in comparison withthe speed of the eye movement, however, the distortions differ onlyslightly in the case of images which are rapidly scanned sequentially,so that they can be roughly neglected and the two-dimensional images canbe used directly without taking into account the different distortions.When the orientation of the eye changes noticeably, the distortions inthe various two-dimensional images may also clearly differ from oneanother so that such images can, for example, no longer be compared withone another in a suitable manner. It may therefore be advantageous toagain scan certain images at certain time intervals or in the event thata clear orientation change is detected.

4.2 Strategy

By means of the detected picture elements, the position of certaincharacteristic features or structures of the part of the eye, whichreflects the detected light, can be determined within the image.

For this purpose, the eye can be scanned along a suitable trajectory.This trajectory may, for example, be selected such that particularlysignificant characteristics, for example, changes of the reflectivity orthe like, are detected with high probability.

For example, the position of the pupillary center and/or of the maculacenter can be determined. This may be relevant for determining thedirection of vision.

Several of the described methods can also be carried out successively,in which case the information obtained in a preceding method can be usedfor subsequent methods. As an example, the pupillary center can first beroughly determined. The thus found approximate center is stored and isused as a starting or reference value for the precision determination ofthe actual pupillary center. The approximate or the actual pupillarycenter can, in turn, serve as a starting value or preliminary maculacenter when determining the macula center because, with a certainprobability, for example, the macula center may be situated in a certainposition with respect to the pupillary center. This position may beknown or estimated, for example, empirically from statisticalinvestigations and/or preceding determinations according to theinvention. For example, in the direction of vision of the eye, with ahigh probability, the macula center may be situated directly behind thepupillary center.

In particular, all methods described above and below can be “readjusted”by means of already identified characteristics; i.e., already identifiedcharacteristics can be used as reference and starting values forsubsequent methods. For this purpose, a reference point, for example,can be determined, and then one or more optical devices, thus, forexample, the detector light-guiding device and/or the projectionlight-guiding device can be aligned with respect to this referencepoint.

Advantageously the scanning can be carried out at a high scanningfrequency. As a result, the orientation of the eye between two or moresuccessively scanned images advantageously changes only little. As aresult, the position of significant characteristics within the scannedimages also changes little, which can advantageously facilitate thesearch for these characteristics.

4.3 Analysis

When, after at least one of the above-mentioned processes, the two- orthree-dimensional position of one or more significant characteristics ofthe eye has been determined, orientations of the eye relative to thedevice and/or the environment can be determined therefrom in variousmanners. Here, it should be pointed out that orientation may alsocomprise the determination of a change of the orientation with respectto an orientation selected as a reference.

For the purpose of an explanation, for example, in a simple case, theorientation of an eye at a certain point in time may be selected as areference, and the position of the pupillary center may be determined inan image of the light reflected by the sclera or the iris. When, at alater point in time, the position of the pupillary center is againdetermined in an image of the light reflected by the sclera or the iris,a corresponding rotation of the eye can be determined from the newposition of the pupillary center relative to the old position. When, inaddition or as an alternative to the position of the picture element,for example, the distance from the detector is determined, the positionof the element and thus of the eye relative to the device according tothe invention can also be determined therefrom.

In the following, several possibilities will be explained as examplesfor the determination of the orientation, particularly of the directionof vision.

4.3.1 Determination of the Visual Axis

A straight line through the center of rotation of the eye and the centerof the pupil, for example, is assumed to be the direction of vision orthe visual axis.

When the position of the pupillary center is determined for differentpositions of the eye, the images of the pupillary center in atwo-dimensional scanned image are situated within a circle (or becauseof the distortion by the two-dimensional imaging of thethree-dimensional eye, within an ellipse) around the imagined image ofthe center of rotation which can thereby be determined (for example, asa statistical center of the images of the pupillary center). In asimplified manner, the straight line through the center of rotation andthe pupillary center then represents the visual axis of the eyeaccording to the assumption. For the purpose of clarification, a purerotation of the eye should, for example, be considered about a verticalaxis through the center of rotation. The two-dimensional image of thepupillary center in this case moves on a horizontal straight line in animage plane perpendicular to the viewing direction and parallel to thevertical axis of rotation. In the case of very many taken pictureelements, the main emphasis of this straight line correspondsapproximately to the image of the center of rotation.

Likewise, for example, from the distortion of the pupil, which appearsas an ellipse in the case of a non-frontal top view, the plane of thepupil can be determined by a principal axis transformation; themid-perpendicular of the plane of the pupil can be assumed to be thevisual axis.

Likewise, the visual axis can be determined as the connecting straightline of the fovea centralis and the pupillary center. Thethree-dimensional position of the pupillary center and the foveacentralis can be estimated, for example, by means of empirical data fromthe position of the pupillary center and the position of the foveacentralis relative thereto. For this purpose, stored values, forexample, can be used for this position, which are known from earlierinvestigations (for example, the so-called Gullstrand values) or valueswhich were determined by means of a device according to the invention.Likewise, as described above, the distance of the pupillary centerand/or of the fovea centralis from the detector can be determined bymeans of the propagation time of the reflected light beam between theemission and the detection and/or by means of the focusing and, togetherwith the two-dimensional position of the picture elements, thethree-dimensional position of the pupillary center and/or of the foveacentralis can be determined therefrom.

4.3.2 Determination of the Orientation by Means of a Map

For determining the orientation of the eye, according to the invention,a reference map of a part of the eye can be established in whichsignificant structures, such as the macula, blood vessels of the retina,injuries of the cornea or the like, can be recognized. Thesecharacteristics can advantageously be entered on a correspondingreference map by means of image or pattern identification.

A reference image and/or a reference map may be implemented, forexample, by means of a memory, in each of the memory locations, thevalue of the characteristic quantity of a detected light beam assigned,for example, according to the position, being stored. Advantageously,the characteristic values may previously have been filtered such thatmemory locations are only occupied by a defined characteristic value,for example, a value higher than a defined threshold value. In areference map, significant structures of the scanned reference image mayalready be imaged and stored with a low storage requirement.

When now, at a later point in time, according to the invention, a partof the eye is scanned on which the structures detected as describedabove are at least partially present—advantageously the same area asduring the establishment of the reference image—points of an actualimage detected in such a manner can be caused to be congruent with thereference image or the reference map by means of a correspondingroutine. From the resulting rotations and displacements of the actualimage, the change of orientation of the eye with respect to the point intime of the taking of the reference image can be determined. In thiscase, the actual image can be taken by means of a different trajectorythan the reference image. In particular, it may clearly have fewerpicture elements. Here, it is particularly advantageous that causing thecongruency of a few actual picture elements with a few reference pointscan permit a fast and simple determination of the orientation. Likewise,by means of the actual image, an actual map can be established and becaused to match the reference map, the change of the orientation of theeye again being determinable from the occurring rotations anddisplacements.

Advantageously images can be scanned at a high scanning frequency, sothat the orientation of the eye changes little between two or moresuccessive scannings. This can advantageously simplify the comparison ofan actual image with a reference image or a reference map because theposition of the picture elements of the same point in or on the eye haschanged little, and the correlation between the picture elements ofdifferent scanning points in time can be easy to establish.

As described above, it may be advantageous to again detect(scan?—transl.) a reference image or a reference map when it is foundthat the orientation of the eye has changed so much that, as a result ofthe distortion because of the two-dimensional imaging of athree-dimensional structure, matching becomes difficult or impossible. Acorrespondingly severe change of orientation of the eye can bedetermined, for example, in that it is no longer possible to cause anactual image and the reference image or the reference map to becongruent.

4.3.3 Orientation with Respect to the Environment and/or the Device

According to the invention, generally the absolute and/or relativeposition and/or orientation, particularly the direction of vision, of aneye relative to the environment and/or to the device according to theinvention can be determined. This results in different possibilities ofdetermining the orientation, of which some will be explained in detailin the following.

4.3.3.1 Relative Kinematics

A number of possibilities were explained above in the manner of exampleswith respect to determining the orientation of the eye relative to thedevice and particularly the direction of vision of the eye by means ofone of more images which can be created by detecting light beamsreflected by a part of the eye.

When, as indicated above, in addition, the position of the devicerelative to the environment is determined, for example, by means ofmagnetic, IR or RF position finding of the device from an inertiallyfixed device or by means of a GPS receiver on the device according tothe invention, the orientation of the eye with respect to theenvironment can be kinematically calculated from the relativeorientation of the eye with respect to the device and the relativeposition of the device with respect to the environment.

Inversely, when the orientation of the eye relative to the environmentis known and the orientation of the device relative to the environmentis known, the orientation of the eye with respect to the device isobtained analogously. When, for example, the orientation of the eye withrespect to the environment is described by means of the angles (Φ_(AU),η_(AU), φ_(AU)), the orientation of the device with respect to theenvironment is described by means of the angles (Φ_(VU), η_(VU),φ_(VU)), and the orientation of the eye with respect to the device isdescribed by the angles (Φ_(AV), η_(Av), φ_(AV)), the orientationsexpressed by the respective imaging A_(xy) from x to y can be changedinto one another:

A _(Au) =A _(VU)(Φ_(VU),η_(VU),φ_(VU))*A _(Av)(Φ_(Av),η_(AV),φ_(Av))

A _(AV) =A _(VU)(Φ_(VU),η_(VU),φ_(VU))*A _(AU)(Φ_(AU),η_(AU),φ_(AU))

The second equation therefore indicates, for example, that the imagingof a coordinate system fixed to the eye onto a coordinate system fixedto the device and thereby the orientation of the eye relative to thedevice can be determined from a suitable linking of the imaging from thecoordinate system fixed to the eye onto a coordinate system fixed to theenvironment and an imaging of this coordinate system fixed to theenvironment onto the coordinate system fixed to the device. These twoimagings can be determined by the orientation of the eye relative to theenvironment and of the device relative to the environment.

In the following, images are compared with one another which are fixedwith respect to the environment (environment image), with respect to adevice according to the invention (device image) or with respect to theeye (eye image). These images are preferably compared with one anotherin a common coordinate system, for example, a coordinate system fixedwith respect to the device; i.e., for example, at least two imagesrespectively are illustrated in this coordinate system and significantstructures in the images are assigned to one another. This means, forexample, that their relative position and/or orientation with respect toone another is determined. Analogous to the “orientation” of an eye, the“position” of characteristics, structures, etc. comprises the positionin a narrower sense, characterized, for example, by distances fromcoordinate axes, and/or their orientation in a narrower sense as in theremaining application, characterized, for example, by angles withrespect to coordinate axes.

By means of the change of these relative positions, the change of thepertaining images with respect to one another can be determined in eachcase. From this change, the corresponding orientation change of thepertaining systems (eye, device, environment) with respect to oneanother can in each case be determined. For example, from a displacementof an eye image with respect to a device image, a correspondingdistortion of the eye with respect to the device can be determined.

As an alternative or in addition, an absolute assignment of thepertaining systems can also be determined from the relative positions.For example, from the relative position of a significant structure in aneye image (for example, of the macula) with respect to a significantstructure in an environment image (for example, an edge, a tree or thelike), the position of the eye with respect to the environment can bedetermined. Thus, it can, for example, be determined where the eye islooking relative to the significant structure in the environment (forexample, directly at the edge, at the tree or the like).

In the following, some of these possible determinations will beexplained in detail.

4.3.3.2 Eye—Environment

When an image fixed relative to the environment (environment image),such as an image of the environment which is taken by a camera fixedlyconnected with the device, is compared with an image fixed relative tothe eye (eye image), such as an image of the retina or of the corneawhich is, for example, a result of an active scanning, an orientation ofthe eye relative to the environment can be determined therefrom. Whenimages taken at different times are compared, a change of orientation ofthe eye relative to the environment can be determined therefrom.

4.3.3.2.1 Relative Orientation Eye—Environment

For determining the orientation of the eye relative to the environment,an environment image and an eye image are viewed in a common coordinatesystem, for example, in a coordinate system fixed with respect to thedevice (device-fixed coordinate system).

In this case, the images can be appropriately imaged in this coordinatesystem. For example, from the known geometry of the arrangement of acamera, which supplies the environment image, with respect to thedevice, which supplies the eye image, it can be determined how theenvironment image is to be transformed into a device-fixed coordinatesystem of the eye image. Likewise, for example, a significant shape ofthe device, for example, the geometrical shape of a pair of spectacles,may be recognizable in the environment image. By means of thesignificant shape of the device in the environment image, it can then bedetermined how the environment image is to be transformed into thedevice-fixed coordinate system of the eye image.

Advantageously, an image of the environment reflected by a part of theeye can be used as the environment image, such as the environment refleximage of the retina or the cornea because the environment image and theeye image are then already present in the same device-fixed coordinatesystem.

These images are compared with one another; i.e., certain significantstructures within the environment image (edges, objects, letters, etc.),which can be identified, for example, by means of patternidentification, are assigned to certain significant structures of theeye image (for example, the macula, the pupillary center, etc.), andtheir position relative to one another is determined.

By means of this relative position, the orientation, particularly thedirection of vision, of the eye relative to the environment can bedetermined.

For example, it can be determined in the eye image at which point themacula center is situated. This point can be determined in theenvironment image. It can then be determined that the viewer's eye islooking specifically at this point in the environment. As a result, thedirection of vision of the eye relative to the environment can bedetermined. Advantageously, a pattern identification can identify theobject in the environment image at which the viewer is just looking, forexample, which switch, which target or which object.

Advantageously, an environment image which is taken by means of a camera(camera picture) can be correlated with an environment image which isreflected by the eye and scanned (environment reflex image). The twoimages can be assigned to one another such that a point of theenvironment which is defined in one image can be determined in the otherimage. For example, in a relatively blurred and/or low-light environmentreflex image, by means of the method according to the invention, thepoint of the environment can then be determined at which the eye islooking, and this point or the pertaining object, switch, etc., can bebetter identified in a more focused and/or more luminous camera pictureby means of pattern identification. This is, for example, advantageousin the case of diffuse light at which an infrared camera can scan a moreprecise image of the environment.

4.3.3.2.2 Relative Orientation Change Eye—Environment

Likewise, a change of the orientation of the eye relative to theenvironment can be determined in that, analogous to a method accordingto the invention, at different points in time, in each case, oneorientation of the eye with respect to the environment is determined,and the change between the orientations determined at the differentpoints in time is indicated.

For example, significant characteristics in the environment image(edges, etc.) and in the eye image (macula, etc.) can be assigned to oneanother. For example, both images may in each case be superimposed toform a total image such that their image boundaries are arranged at afixed ratio to one another (in the case of environment and retina refleximages of the same size, for example, congruently). This means that thetwo images can be represented in a common coordinate system, preferablyin a coordinate system that is fixed with respect to the deviceaccording to the invention. In the total image, distances or relativepositions between significant characteristics of the environment imageand the eye image can be determined. From the change of the relativeposition of the significant characteristics with respect to one anotherin the total image, the change of the orientation of the eye withrespect to be environment can be determined.

Likewise, the two images (for example, by means of a correlation methodor by rotation and displacement) can be made congruent. From theresulting values (such as correlation factors or angle of rotation anddisplacement), the change of orientation of the eye with respect to theenvironment can again be determined.

4.3.3.3 Orientation Device—Environment

An orientation change of the device with respect to the environment canbe determined from the change of an image fixed relative to theenvironment (environment image), for example, of an image taken by acamera fixedly connected with a device, or reflected by the retina orthe cornea and detected by a device according to the invention.

For this purpose, for example, two environment images taken at a timeinterval from one another are compared with one another in the samecoordinate system preferably fixed with respect to the device. When theorientation of the device with respect to the environment has changed inthe interim, the same significant patterns within the environment imageat different points in time have different positions within thecoordinate system. This results in the change of orientation. As analternative, the two images can be made congruent (for example, by acorrelation method or by rotation and displacement). From the resultingvalues (for example, correlation factors or angle of rotation anddisplacement), the orientation change of the device with respect to theenvironment can again be determined.

Likewise, the orientation of the device with respect to the environmentcan also be determined by a comparison of a device image with anenvironment image in that, for example, both images are viewed in acommon coordinate system preferably fixed to the device, and significantstructures in the device image and in the environment image are assignedto one another. An orientation of the device relative to the environmentcan then be determined from the relative position of these structureswith respect to one another, for example, of a marker on the device orthe geometrical shape of a pair of spectacles, with respect to asignificant edge, a pattern, etc. in the environment image.

By a repeated time-shifted ascertaining of such an orientation, anorientation change of the device can also be determined with respect tothe environment.

When, in addition, the orientation of the eye with respect to the deviceis determined, relative-kinematically, an orientation change of the eyewith respect to the environment can be determined therefrom.

Likewise, when the orientation of the device with respect to theenvironment is known, which may be determined, for example, by means ofan orientation determining device, and the orientation of the eye withrespect to the device is known, relative-kinematically, an orientationof the eye with respect to the environment can be determined.

4.3.3.4 Orientation Eye—Device

By means of an image fixed with respect to the eye (eye image), forexample, an image of the retina or of the cornea, in which structures ofthe eye, such as scars, blood vessels, etc. can be recognized, theorientation of the eye can be determined with respect to the device.

When, in addition, the orientation or change of orientation of thedevice with respect to the environment is known, together with theorientation or orientation change of the eye with respect to the device,relative-kinematically, the orientation or orientation change of the eyecan be calculated with respect to the environment.

4.3.3.4.1 Relative Orientation Eye—Device

4.3.3.4.1.1 Markers

When an image fixed relative to the device (device image), for example,an image in which a marker fixed relative to the device can berecognized, is compared with an image fixed relative to the eye (eyeimage), for example, an image of the retina or of the cornea, which isobtained, for example, from an active scanning, an orientation of theeye relative to the device can be obtained therefrom. When images arecompared which were taken at different points in time, an orientationchange of the eye relative to the device can be determined therefrom.

For determining the orientation of the eye relative to the device, adevice image and an eye image can be considered in a common coordinatesystem, preferably in a coordinate system fixed respect to the device(device-fixed coordinate system).

Advantageously, an image reflected by a part of the eye can be used as adevice image, for example, the environment reflex image of the retina orof the cornea or a retina reflex image, in which a marker can berecognized which is fixed with respect to the device. Particularlypreferably, structures fixed with respect to the device (marker,geometrical shape of the spectacle lens, etc.) within the eye image canbe used as a device image, because then the device image and the eyeimage are already present in the same device-fixed coordinate system oradvantageously are already superimposed in the same image.

These images are compared with one another; i.e., certain significantstructures within the device image (marker, edges of the spectacle lens,etc.), which may be identified, for example, by means of patternidentification, are assigned to certain significant structures of theeye image (for example, the macula, the pupillary center, etc.), andtheir position relative to one another is determined.

By means of this relative position, the orientation of the eye relativeto the device can be determined.

For example, it may be determined in the device image at which point amarker is situated. In the eye image, it may be determined at whichpoint the macula center and/or pupillary center is situated. By means ofthis relative position, the orientation of the eye relative to thedevice can be determined.

The orientation change of the eye relative to the device can thereforebe determined when a significant pattern of the device can be recognizedin a scanned image of a part of the eye.

For this purpose, for example, a marker may be present on an opticaldevice of a device according to the invention, for example, on a mirror,on a lens, in a hologram, on a spectacle lens in front of the eye, etc.,in such a manner that light beams are reflected particularly strongly(“light” marker) or particularly weakly (“dark marker). The marker may,for example, be generated holographically, preferably in a wavelengthrange not perceived by the eye, for example, in the infrared range. Anedge of a spectacle lens, for example, may also be used as a marker.

A detector detects light beams which are reflected by a part of the eye,for example, the thermal radiation of blood vessels of the retina orlight reflected by the retina. In an image created therefrom,significant structures of the eye can be detected which are fixedrelative to the eye, for example, blood vessels or the macula. In theimage, the marker fixed relative to the device is also visible, which isarranged, for example, on a fixed mirror of a detector light-guidingarrangement such that a light beam reflected from the eye, whichimpinges on the fixed mirror at the point of the marker, is not guidedor is guided only very little in the direction of the detector. In thedetected image, the marker will then appear dark. From the relativeposition of the images of at least one significant structure of the eyerelative to an image of the marker, the orientation of the structure andthus of the eye relative to the marker and thus to the device can bedetermined.

Further markers according to the invention and their use respectivelyare disclosed in the “Marker” section in the “Optical Device” section.

4.3.3.4.1.2 Position of the Device

For determining the orientation of the eye relative to the device, aneye image can be viewed in a coordinate system fixed with respect to thedevice (device-fixed coordinate system).

For example, a coordinate system fixed to the device may be determinedby a neutral position of the detector system. In an eye image viewed insuch a coordinate system, certain significant structures (macula center,pupillary center, etc.) may be determined, and their position withrespect to the coordinate system may be determined. By means of thisposition, an orientation of the eye relative to the device can then bedetermined.

4.3.3.4.2 Relative Orientation Change—Device

Likewise, a change of the orientation of the eye relative to the devicecan also be determined in that, analogous to an above-described method,at different points in time, one orientation of the eye respectively isdetermined with respect to the device.

Thus, for example, for this purpose, the relative position of at leastone defined significant structure of an eye image, which was scanned atdifferent points in time, can be determined within a device-fixedcoordinate system and/or relative to a device image, particularly, forexample, a marker. From the change of this relative position between thedifferent points in time of the scanning, a change of orientation of theeye with respect to the device can then be determined.

Likewise, two eye images, which were scanned at different points time,(for example, by a correlation method or by rotation and displacement)can be made congruent. From the resulting values (for example,correlation factors or angle of rotation and displacement), theorientation change of the eye with respect to the device can again bedetermined.

Full Content Reference to Other Applications, Disclosures

As explained in this specification, the present invention canadvantageously be used in connection with the systems, devices andmethods described in Published Patent Applications or ApplicationsPCT/EP00/09843, PCT/EP00/09840, PCT/EP00/09841, PCT/EP00/09842, DE 10127 826, PCT/EP01/05886, DE 196 31 414 A1 and DE 197 28 890 A1. Thepresent invention can also advantageously be used in connection with theapplication with the title “Information System and Method for ProvidingInformation by Using a Holographic Element” filed by the applicant ofthis application on Oct. 8, 2001. The entire content of these publishedpatent applications and applications respectively is thereforeexplicitly included in this application by reference.

PREFERRED EMBODIMENTS

The determination of the position and/or orientation, particularly ofthe direction of vision, of the eye preferably takes place without beingperceived. As a result, it is avoided that the user may be disturbed byvirtual points in his field of view.

A device according to the invention is preferably constructed to beportable, particularly in the form of a pair of spectacles. It therebybecomes possible, for example, to also use this device in automobiles,airplanes, etc and in each case determine the direction of vision of theeye.

A projection system according to the invention preferably projects aninfrared light beam onto the eye, the diameter of the light beam beingvery small in comparison with the diameter of the pupil. Preferably, theocular, particularly the retinal reflex of the beam is detected. Incontrast to a prejudice in the state of the art, a light beam with sucha small diameter is sufficient for generating a sufficiently reflectedbeam. When scanning by means of light beams of small diameters, thecurvature of scanned parts of the eye advantageously does not play asignificant role and especially does not cause any significantdistortions of the image.

A device according to the invention preferably comprises a splittermirror arranged in an infrared light beam, which allows only a smallfraction of the infrared light beam to pass and reflects acorrespondingly large fraction of the impinging ocular reflex in thedirection of a detector device. As a result, light is projected on thesame beam path onto or into the eye or reflected from or out of the eye,whereby it can, for example, be ensured that the detector alwaysreceives a signal and does not first have to be adjusted to theilluminated point.

A projection system according to the invention preferably projects lightin a pixel-type manner with a defined pixel frequency onto the eye.Particularly preferably, a projection system according to the inventionmodulates the projected light with a frequency higher than the pixelfrequency. As a result, it becomes advantageously possible to transmitinformation within a light beam reflected at a point or pixel, forexample, concerning the point in time of the emission or in order to beable to differentiate the light beam from the ambient light.

Preferably, no active illumination of the eye takes place, and thedetector system carries out a pixel-type scanning of the ambient lightreflected back from the eye and/or of the light emitted by the eye.Advantageously, no projection system is required in this case.

Preferably, a device according to the invention has a surface that canbe positioned in front of the eye: the marker areas which reflect animpinging projection beam originating from the projection systemcompletely back in the direction of the detector system, as well asnormal areas which guide an impinging projection beam originating fromthe projection system in the direction of the center of the eye. Bymeans of the marker area, for example, the orientation of the eye can bedetermined with respect to the marker areas and thereby the deviceaccording to the invention.

A device according to the invention preferably determines the positionand/or orientation of the eye with respect to its environment in thatthe detector system detects the retinal structure of the eye as well asalso the environment reflex image superimposed thereon, detects theposition of the fovea by means of the retina structure and identifiesthe area of the environment sighted by the fovea by means of a patternidentification.

The device according to the invention preferably detects arepresentation at least of selected area of the retina and files thesein an intermediate memory. For determining a change of the spatialposition of the eye, the device according to the invention particularlypreferably compares the filed representation with information which thedevice has obtained from light scanned from the retina and detectedduring an actual scanning movement. This may advantageously permit afast comparison.

The device according to the invention preferably comprises a surfacehaving a predefined geometrical shape which can be positioned in frontof the eye and by way of which light can be projected from theprojection system into the eye, the geometrical shape of the surfacebeing used for determining the relative position of at least onecharacteristic area of the retina with respect to the optical detectorsystem and/or projection system. Advantageously, no additional markersare required in this case.

EMBODIMENTS AND EXAMPLES OF APPLICATION

In the following, some of the characteristics or terms indicated ingeneral above will first be explained in detail or by means ofembodiments of the present invention. For a better clarity, reference ismade in this case to the structuring of the above summary of theinvention. It is pointed out expressis verbis that the followingspecifications have the purpose of illustrating examples and representno restriction of the characteristics or terms generally specified inthe summary. Some preferred embodiments of the present invention willthen be indicated.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a view of an embodiment of the device according to theinvention in the form of spectacles;

FIG. 2 is a longitudinal sectional view of a human eye;

FIG. 3 a is a view of a course of the radiant intensity I_(e) over thetime;

FIG. 3 b is a view of another course of the radiant intensity I_(e) overthe time;

FIG. 4 is a view of a controllable laser as the radiator;

FIG. 5 is a view of an arrangement of the radiator and the detector witha common light-guiding device;

FIG. 6 is a view of an embodiment of the device according to theinvention in the form of spectacles;

FIG. 7 a is a view of an embodiment of a device according to theinvention in the form of spectacles;

FIG. 7 b is an enlarged view of the radiator-detector combination fromFIG. 7 a;

FIG. 8 is a view of a movement pattern for determining the pupillarycenter;

FIG. 9 is also a view of a movement pattern for determining thepupillary center;

FIG. 10 is a view of a movement pattern for determining the maculacenter;

FIG. 11 also is a view of a movement pattern for determining the manularcenter;

FIG. 12 is a schematic view of the retina with a spiral-shaped scanningpattern;

FIG. 13 is a view of a scanned image of the retina from FIG. 12;

FIG. 14 is a schematic view of the retina with old and new axes;

FIG. 15 is a schematic view of an eye with the pupil in a reference androtated position a well as the corresponding scanned image;

FIG. 16 is a schematic view of an eye in the reference and rotatedposition as well as an environment;

FIG. 16A is a view of the environment reflex image of the retina;

FIG. 16B is a view of a retina reflex image;

FIG. 16C is a view of an environment reflex image of the retina when theeye is rotated;

FIG. 16D is a view of the retina reflex image when the eye is rotated;

FIG. 16E is a view of the environment reflex image of the retina whenthe device is rotated;

FIG. 17A is a view of a spectacle lens with a marker in front of an eye;

FIG. 17B is a view of an environment reflex image and a superimposedretina reflex image of the eye from FIG. 17A;

FIG. 17C is a view of the arrangement from FIG. 17A when the eye isrotated;

FIG. 17D is a view of an environment reflex image and a superimposedretina reflex image of the eye from FIG. 17C;

FIG. 18 is a view of a reflected principal ray and of marginal rays.

DETAILED DESCRIPTION OF THE INVENTION

A device and a method respectively for determining the position and/ororientation of an eye according to the present invention consist ofdetecting and analyzing a light signal reflected by a part of the eye bymeans of a detector system.

For a better understanding, this will be explained as an example bymeans of an embodiment of the present invention. For this purpose, FIG.1 illustrates a device according to the invention in the form ofspectacles 101 on a user's head 102, the right half of the figurerepresenting the top view of the head 102, and the left half of thefigure representing a sectional view approximately at the level of thecenter of the eye. Light beams 103 a, 103 b (“light signal” in thepresent embodiment) enter the user's left eye 104 from the environmentand are partially reflected on its retina 105 (“part of the eye” in thepresent embodiment). A light beam 106 reflected at a certain point ofthe retina impinges on a first fixed concave mirror 107 and is guided byit onto a second fixed concave mirror 108. From this mirror 108, thebeam 106 is guided onto a horizontally movable flat mirror 10911 and isguided by the latter onto a vertically movable flat mirror 109V. Fromthe vertically movable flat mirror 109V, the beam 106 is guided onto aphoto detector 110 which detects the incident beam 106 with respect toits intensity, wavelength or the like (in the present embodiment,“detector system” comprises the photo detector as well as the mirrorarrangement). By suitable movements of the flat mirrors 10911 and 109Vrespectively, light beams, which were reflected at different points ofthe retina, can be sequentially guided into the photo detector 110. Inthis manner, the retina can be scanned along a predefined movementpattern (scan pattern). From the resulting information, an orientationof the eye can be determined (“Analysis” in the present embodiment).

For a better understanding, the operating method and the technicalterminology of the eye will be briefly discussed in the following.

The eye or the eyeball 280 illustrated in FIG. 2 is moved by the sixmuscles 24 in the eye socket (orbita) 20 and, as a result, itsorientation relative to the skull is adjusted. The upper eyelid 27 a andthe lower eyelid 27 b with the eyelashes 27 c change over into theconjunctiva 26 which closes off the space between the orbita 20 and theeyeball 280.

The eyeball 280 consists of a transparent, approximately sphericalvitreous body 21, on which a deformable lens 282 rests in the front inthe direction of vision, the focal point of the lens 282 beingchangeable by the tensioning and relaxing of ciliary muscles 23 arrangedon the circumference. Directly in front of the lens 282, an aperture(iris) 285 consisting of a colored material is arranged whose pupil 284can be changed in its diameter in a muscular manner.

The rearward part of the vitreous body is surrounded by the sclera 28whose interior side is covered by the choroid 287. Between the vitreousbody 21 and the choroid 287, the retina 281 is situated which issupported by the vitreous body and supplied with blood by the choroid.The retina comprises extremely light-sensitive rods for the scotopicvision (in twilight) as well as less light-sensitive cones for thephotopic vision (color vision in daylight). The so-called yellow spot(macula) with the fovea centralis 286—the region with the sharpestvision—with a very high cone density is situated inside the retina. Theblind spot 288 is situated in another region, which blind spot 288, theoptic nerve 25 leads into the retina and where no image information canbe received.

The forward part of the vitreous body is surrounded by the transparentcornea 283 such that the anterior chamber 22 of the eye is formedbetween the cornea and the iris and is filled with a transparent liquid.At its circumference, the cornea changes over into the sclera.

In the optical system of the eye, light which is incident in parallel inthe case of an emmetropic relaxed eye is essentially focused through thecornea on the retina. By changing the refraction of the lens 282, objectat different distances are imaged sharply. In the following, themid-perpendicular onto the principal plane 290 of the lens 282 is calledthe optical axis 291 of the eye 280. In contrast, the straight linethrough the pupillary center and the fovea centralis 286 is defined asthe direction of vision 292 because, as a rule, a human being will focushis eye on a point to be viewed such that this point is imaged in theregion of the sharpest vision. While, in the normal case, the opticalaxis 291 and the direction of vision or visual axis 292 hardly deviatefrom one another, there is definitely the possibility that the foveacentralis is not situated opposite the pupil. The eye then seems to bestrabismal.

Concerning 1.2 Passive Scanning

From the polyspectral ambient light, light beams of particularlysuitable wavelengths can be filtered out, such as light in thegreen-yellow range of the sharpest vision between 500 nm and 600 nm inorder to thus be able to detect the macula particularly well, or lightin the range of approximately 750 nm, at which the reflectivity of theretina is the highest.

Concerning 1.3.1.1 Amplitude Modulation

Amplitude modulation is a modulation at which at least one of theabove-indicated characteristic quantities in different time segments orat different points in time each have certain values. As an example,FIG. 3 a shows the radiant intensity I_(e) of an actively beamed-inlight over the time, which, in a first time period T1, has a referencevalue of 100% and, in a subsequent second time period T2, has a value of0%, periods T1 and T2 following one another periodically. When such anactively beamed-in light, together with the ambient light, is reflectedat a point of the retina, and the radiant intensity of the entire lightbeam reflected at this point is detected in a detector, approximatelyexactly the actively beamed-in light without ambient light fractions isobtained from the difference of the light intercepted in period T1 andof the light intercepted in period T2.

Concerning 1.3.1.2 Frequency Modulation

The change of a periodicity of at least one of the above-indicatedcharacteristic quantities is called frequency modulation. Again as anexample, FIG. 3 b shows the radiant intensity I_(e) of an activelybeamed-in light over the time, which is beamed in a first time period T1in a pulsed manner with a period λ1 and in a subsequent second timeperiod T2 in a pulsed manner with a period length λ2≠λ1. When such anactively beamed-in light, together with the ambient light, is reflectedat a point of the retina and detected in a detector, it can bedetermined by means of the period λ1 or μ2 whether the light beam wasreflected during period T1 or period T2. Generally, for example, theperiodicity between a not shown point in time TA and a not shown pointin time TE can be changed according to a predefined relation; thus, forexample, decrease in a linear fashion between TA and TE. Then, from theperiodicity of the reflected and detected light beam, the point in timebetween TA and TE can be determined at which the beam was emitted. Thispermits, for example, a propagation time measurement and thus, forexample, when the geometry between the detector system or projectionsystem is known, the determination of the distance between thereflecting point of the eye and the detector or projection system as aproduct of the speed of light and of the propagation time. Likewise, itmakes it possible to differentiate between actively beamed-in light andambient light.

One or more characteristic quantities of the actively beamed-in lightbeam may simultaneously or sequentially be amplitude-modulated and/orfrequency-modulated. In FIG. 3 b, the period length (frequencymodulation) as well as the pulse width (pulse code modulation) ischanged between period T1 and period T2.

Concerning 1.3.2 Adaptation of the Active Luminous Intensity

In contrast to the passive scanning, the active scanning has theadvantage that it can be ensured that always light of a sufficientintensity is reflected from the eye in that actively a correspondinglylarge amount of light is beamed in. For example, the radiant intensityof the radiator, which actively emits the light, can be adapted suchthat the detector can sufficiently clearly detect the resultingreflected light beam. When the detector can no longer detect theactively beamed-in and reflected light beam, the light intensity can beincreased correspondingly. When the detector records a very high lightintensity of the actively beamed-in and reflected beam, the lightintensity is correspondingly reduced. The orientation of the eye cantherefore also be determined in darkness (virtual-reality spectacles,night vision device, etc.) or under considerably changing lightconditions (driving through a tunnel, etc.).

Concerning 1.3.3.1 Point-Focal Illumination

According to the invention, a light beam can be emitted from a radiator,for example, a laser and can be guided by means of a projectionlight-guiding arrangement such that successively, i.e, sequentially,different points on the part of the eye are illuminated. This can becarried out, for example, by means of a device which is constructedanalogously with respect to the device illustrated in FIG. 1, thedetector 110 being replaced by a radiator (reversibility of the beampath). By means of a corresponding movement of the two flat mirrors 109Hor 109V, the light beam emitted by the radiator is then sequentiallyguided to different points of the retina 105. In this case, the endpoint of the light beam on the eye follows a movement pattern(projection movement).

As an alternative, the radiator itself can guide the emitted light beamonto different points of the part of the eye. For example, acontrollable laser can emit the emitted light beam to a curved ormultipart mirror such that, in each case, a certain mirror area isimpinged upon at a certain angle. The mirror areas are mutually inclinedby different angles, so that a light beam impinging at the same anglewith respect to the entire mirror is deflected on different mirror areasinto different directions. For this purpose, FIG. 4 shows a controllablelaser 401, which can deflect a light beam maximally by the angle α fromthe principal axis 402. In the case of a maximal deflection α in theplane of projection, the marginal rays 403 a and 403 b respectively areobtained. These are reflected to the corresponding mirror areas 404 aand 404 b respectively of a mirror 404, these mirror areas beingappropriately inclined with respect to one another. With respect to theprincipal ray 406 reflected on a mirror area 404 c inclined with respectto the mirror areas 404 a and 404 b respectively, the reflected marginalrays 405 a and 405 b respectively have an angle δ which is greater thanthe original angle α. In this manner, also by means of a radiator whichcan only implement a small deflection of the emitted beam, sufficientlylarge scanning areas can be implemented. The surface of the mirror 404with the corresponding mirror areas 404 a, 404 b and 404 c respectivelyis geometrically obtained from the corresponding angles and distances.

Concerning 1.3.3.1.1 Detection of Light from the Entire Pupil

FIG. 18 illustrates a light beam 1804 which is beamed into an eye 1801having a pupil 1802 and a lens 1803 and which has a very small diameter.This light beam is beamed back (reflected or scattered) at a point 1805of the retina of the eye 1801, for the most part as a principal ray 1806which, in the illustrated case, accidentally corresponds to thebeamed-in light beam 1804. This special configuration is chosen hereonly for the purpose of a clearer representation and does not restrictthe embodiments. The entire light 1804 is not reflected back in thisdirection in parallel as a main ray 1806, but a portion is alsoreflected in other directions as rays 1806 a, 1806 b, 1806 c, 1806 d,resulting, for example, in a lobar intensity distribution of thebeamed-back light around the principal ray. A portion of these lightrays reflected in another than the reflection direction exits throughthe pupil 1802 from the eye and extends approximately coaxial withrespect to the reflected principal ray 1806. For this purpose, the twomarginal rays 1806 b, 1806 c are outlined which just barely still passthrough the pupil 1802. Rays 1806 a, 1806 d are reflected such that theyno longer pass through the pupil 1802. A detector light-guidingarrangement 1807 (in a simplified manner illustrated as a mirror) isconstructed such that also these marginal rays are guided to a detector1808 and are detected by it, whereby the detected quantity of light isadvantageously increased.

Concerning 3.2.1 Mirrors

In a preferred embodiment, the detector light-guiding arrangementcomprises one or more mirrors, whose orientation to one another, to theeye whose orientation is to be determined, and to the detector can beadjusted such that the light beams which were reflected at certainpoints of the eye are sequentially guided to the detector.

The orientation of the movable mirrors can be controlled such that adesired movement pattern can be scanned. As illustrated, for example, inFIG. 1, two flat mirrors may be present for an, as a rule,two-dimensional pattern (such as a grid, a spiral or the like,) whichflat mirrors can be moved about two mutually different axes. As analternative, a wobble mirror, for example, can also be used.

As illustrated in the example of FIG. 1, in addition to the movablemirrors, fixed mirrors may also be present. A first fixed mirror may,for example, be a semitransparent mirror which allows light to pass inthe direction onto the eye and deflects light exiting the eye. As aresult, it becomes possible to arrange the detector outside the visualfield of the eye. Additional fixed mirrors, as also illustrated in FIG.1 by means of examples, may be arranged such that the detector and/orthe movable mirrors can be arranged in a particularly favorable manner,for example, outside the field of vision of the eye and/or at a suitablepoint of the device.

Concerning 3.3.1 Radiator

Advantageously, at least one of the physical quantities of the lightemitted by the radiator can be adjusted. For example, a radiator mayemit light beams having a certain time-related pattern with respect tointensity, wavelength or the like, as outlined by an example in FIGS. 3a, 3 b.

Concerning 3.3.3 Projection Light-Guiding Arrangement

In order to guide light emitted by the radiator onto the region of theeye by which it is to be reflected and subsequently is to be detected inthe detector system, a projection light-guiding arrangement may bepresent into which light enters that was emitted by the radiator andwhich guides this light onto the desired region.

In particular, the detector light-guiding arrangement itself may formpart of the projection light-guiding arrangement in that the lightemitted by the radiator enters in the opposite direction parallel intothe beam path in front, in or behind the detector light-guiding device.

As an example, FIG. 5 illustrates a radiator 501 which emits a lightbeam 502. The latter is guided from a mirror 503 onto a semitransparentmirror 504 which allows the light beam 502 to pass as unchanged aspossible. A semitransparent mirror may be a splitter mirror whichpreferably allows light to pass light through the mirror as unchanged aspossible in one direction and reflects light as completely as possiblein the opposite direction. The light beam 502 then enters into adetector light-guiding arrangement, as formed, for example, in FIG. 1,by the movable flat mirrors 109V, 109H and the two concave mirrors 107,108. From this detector light-guiding arrangement, the light beam isguided and/or focused onto a certain region of the eye, is reflected atthis region and, on the same path, as a reflected light beam 505,impinges again on the semitransparent mirror 504. The latter reflects afraction of the impinging light beam 505 that is as large as possibleonto another mirror 506 which guides the light beam 505 reflected on apart of the eye into a detector 507, where at least one characteristicquantity of this light beam is detected.

Embodiments Concerning the Optical Device

For clarifying the above-indicated characteristics by means of thefigures, several conceivable embodiments of the optical device accordingto the present invention are described by means of a preferredembodiment in the form of spectacles. The invention is not limited tospectacles; it may, for example, also be arranged in a frame wearable onthe head, a helmet, or the like. Likewise, it may, for example, also befastened on a stationary apparatus which uses the information concerningthe orientation of the eye supplied by the device with respect to theorientation of the eye, for example, on an apparatus for laser surgeryor the like. Likewise, it may be fastened on an object which is movablerelative to the environment and relative to the carrier, as, forexample, a portable electronic notebook, a laptop or the like.Furthermore, in the case of the described embodiments, the retina isused as a reflecting part of the eye. As described above, other parts ofthe eye (cornea, iris) can also be used.

In FIG. 6, the user 302 is wearing a device according to the inventionin the form of spectacles 320 having a left and a right bow 321L and321R respectively. The left half of the figure is a top view; the righthalf is a sectional view extending through the left bow 321L.

Light beams 333 a, 333 b are incident in the eye 380 from theenvironment and are focused from its cornea and lens 382 onto the retina381. The retina reflects these beams partially such that the latter exitfrom the eye 380 again through the lens 382. Depending on the wavelengthof the incident light, the reflectivity of the retina amounts toapproximately 0.2-10%. A beam 331 reflected in this manner is guided bya first semitransparent concave mirror 323, which allows light to passas unchanged as possible in the direction onto the eye and almostcompletely reflects light exiting from the eye, and by a second concavemirror 322 onto a horizontally movable first flat mirror 352H, from thelatter to a vertically movable second flat mirror 352V and from thelatter to a photo detector 351. Depending on the position of the twoflat mirrors 352H, 352V, a specific light beam 331, which was reflectedat a certain point of the retina, is guided into the detector 351 and isguided in the latter by means of a focusing device, such as a converginglens, onto a photocell which detects the intensity of the light beam.Intensity of the light beam always applies to a suitable quantity of thereflected light beam measurable by a detector, such as the intensity ofirradiation, the radiance, the light intensity, the brightness or thelike. By means of a corresponding movement of the two flat mirrors,those light beams are therefore sequentially detected which werereflected at points along a corresponding trajectory on the retina; theenvironment reflex image is scanned as a scanning element sequence.

For example, in the case of a fixed first flat mirror 352H and a secondflat mirror 352V uniformly rotated about its horizontal axis, thescanning elements of a vertical straight line are point-focally scannedon the retina. In the case of a first flat mirror 352H uniformly rotatedabout its vertical axis and a fixed second flat mirror 352V, horizontalstraight lines are point-focally scanned on the retina. In a preferredembodiment of the present invention, the two flat mirrors aresinusoidally moved up and down or back and forth, so that the curvescanned on the retina 381 may comprise circles (in the case of the sameamplitude in both mirrors), ellipses (in the case different amplitudesin the two mirrors), spirals (in the case of increasing amplitudes inthe two mirrors) or other suitable curves.

In a further embodiment, the two flat mirrors 352H, 352V are replaced bya wobble mirror movable about at least two axes, which is arranged suchthat it guides light beams from the second concave mirror 322 into thephoto detector 351. By a corresponding controlling of the wobble mirror,scanning element sequences along arbitrary trajectories on the retinacan also be detected by the photo detector 351.

A light trap 324 prevents that light impinges on the eye and/or themirrors from undesired directions of incidence.

Instead of the first concave mirror 323, the spectacle lens 340 can bereflectively coated such that it reflects at least the light beamsreflected by the eye as completely as possible. In order to, also in thecase of rotated eye positions, appropriately deflect light beamsreflected by eye to the second concave mirror 322, the reflectivesurface of the spectacle lens facing the eye should have a correspondingshape. To this extent, reference is made to the full content of PCTApplication “Information System and Method for Providing Information byUsing a Holographic Element” filed by the same applicant on Oct. 8,2001. The reflective surface of the desired shape can preferably beemulated by means of a correspondingly constructed holographic coatingof the spectacle lens 340.

Instead of the different reflective arrangements described here and inthe following (flat mirrors, wobble mirrors or concave mirrors,reflective surfaces, holographic coatings, etc.), in embodiments of thepresent invention, all other known optical refraction or reflectionmechanisms, for example, electro-holographic mechanisms or the like, canalso always be used for guiding the light beams from the radiator to theeye or from the eye to the detector.

In the case of a device for the active scanning, a radiator isadditionally present which emits light, such as a laser, an infraredlamp, a light emitting diode or the like. In an embodiment, the photodetector 351 is arranged in the center of a radiator. Thus, for theprojection of light onto the eye, the same devices (flat or wobblemirror 352H, 352V; second concave mirror 322; first concave mirror 323or reflectively coated spectacle lens 340 or holographically coatedspectacle lens 340) and thus the same beam path is/are used as for thedetection of the reflected light beams. Such an arrangement has theadvantage that possibly present systematic defects as a result ofdeviations of the mirror surfaces or the like are identical inprojection and scanning. When the projection system is used, in additionor as an alternative, for the projection of signals onto the retina, thecorrelation between the scanning point sequence on the retina detectedby the photo detector 351, which may, for example, be the environmentreflex image of the perceived image in the eye, and the projected-ininformation which may, for example, be a virtual marker of an object inthis image, is not impaired by these systematic defects. Furthermore,the common use of the devices reduces the weight of the optical device,and it is ensured that light emitted by the radiator also always returnsat least partly to the detector.

As an alternative, in a (not shown) embodiment of the present invention,an additional second arrangement of a flat or wobble mirror, a secondconcave mirror and a first concave mirror or a reflectively coatedspectacle lens or holographically coated spectacle lens analogous to oneof the above-described embodiments may be present where, instead of thephoto detector 351, a radiator is present, so that reflected light isdetected by the above-described arrangement with the detector 351 andsimultaneously can be beamed into the eye by the second arrangement.

As an alternative, for the active scanning, one of the embodimentsdescribed in the following by means of FIGS. 7 a, 7 a can be used.

FIG. 7 b of a detail illustrates a combined projection and detectorsystem 650L, which is arranged between the left spectacle lens 624L andthe left bow 621L. Naturally, this or another embodiment of the presentinvention may also always as an alternative or in addition beanalogously arranged on the right side, in this case, therefore betweenthe right spectacle lens 624R and the right bow 621R. In the presentembodiment, the left spectacle lens 624L, on the side facing the eye, isprovided with a holographic coating 623L which, as described above,emulates a reflection surface or focusing surface.

The combined projection and detector system 650L comprises a closed-offhousing 658 with a light-transmitting window 659. Although, as a result,light beams can enter into the housing and exit from the housing,disturbing foreign matter, such as dust or the like, can be preventedfrom entering. A radiator 653 and a photo detector 651 are arranged inthe housing 658. A light beam 630 emitted by the radiator 653 impingeson a first wobble mirror 652 which, by means of a suitable drive, ispositioned such that it guides the light beam by means of a splittermirror 656 such onto the holographic coating 623L that the light beam isreflected from there onto a desired point on the retina 681 of the eye680. From this point, the beam 630 is reflected at least partially onthe same path back onto the holographic coating 623L and, from thelatter, on the same path onto the splitter mirror 656. The splittermirror 656 guides a part of the arriving beam 630 onto a second wobblemirror 654 which, in turn, by means of a suitable drive, is positionedsuch that it deflects the beam 630 into the photo detector 651 whichscans the beam 630.

In an advantageous embodiment, the splitter mirror 656 is veryreflective on the side facing the second wobble mirror 654, so that apercentage of the light beam 630 reflected back from the eye, that is ashigh as possible, is guided into the photo detector 651.

Advantageously, a marker 625 is mounted on the spectacle lens 624L. Thismarker can, on the one hand, be used for defining a reference value withrespect to the light conditions and the reflectivity of the individualcomponents in the beam path. For this purpose, by means of the firstwobble mirror 652, the light beam 630 with a known intensity is guidedonto the marker 625 and is partially reflected back by it into theopposite direction. For this purpose, the surface of the marker 625facing the projection and detector system 650L may have a correspondingconstruction, for example, a fully reflective construction. Thereflected beam 630 is deflected on the splitter mirror 656 onto thesecond wobble mirror 654 and by the latter into the photo detector 651,where the intensity is recorded and is defined as a reference value, forexample, for a 100% reflection. It thereby becomes possible to adapt theoptical device also to changing light conditions.

In the case of the present invention, the radiator and/or the detectorcan always be replaced by a light outlet or light inlet device, which isarranged instead of the light outlet or light inlet opening of theradiator or detector and is connected in a light-guiding manner with asuitable optical wave guide, particularly a glass fiber cable or thelike, which, on its other end, is in turn connected in a light-guidingmanner with a radiator or detector. As a result, the radiator ordetector can advantageously be arranged at any point away from the lightguiding arrangement, for example, on the belt of the wearer of a deviceaccording to the invention in the form of spectacles, so that the weightof the projector and/or photo detector does not affect the spectacles.

Concerning 4.1 Detected Image

According to the invention, for the determination, points along atrajectory can always be scanned sequentially, i.e., successively. Theresult is a sequence of scanning points to which, for example, by meansof the position of the detector light-guiding arrangement, certaincoordinates (such as abscissa x and ordinate y; radius R and polar angleΦ) and values for the physical quantity can be assigned. In addition, adistance to the detector can be assigned to each point, for example, onthe basis of a measured propagation time and/or focusing and/orinterference, so that a relation “physical value (x,y)” or “physicalvalue (x,y,z)” is obtained as a “picture”.

For a better understanding, for example, in the arrangement in FIG. 1,an x-value can be assigned to each position of the horizontally movableflat mirror 109H, and a y-value can be assigned to each position of thevertically movable flat mirror 109V. For example, the radiant intensityI_(e) or brightness of a light beam detected by the detector 110 in thisposition of the detector light-guiding arrangement, which light beam wasreflected at a point of the retina 105, can be quantitatively determinedas the physical quantity. When, successively for different positions ofthe two flat mirrors, the brightness of an incident light beam isdetermined and stored, a quantity of value triplets (x, y, is obtainedwhich result in a curve in an x-y coordinate system. By means of acorresponding analog-to-digital conversion, specific brightness orgray-scale values can be assigned to the individual points of the curve.In addition, the propagation time and thereby the distance z of thereflecting point from the detector can be determined, for example, bymeans of a frequency-modulated light beam. A quantity of value tupels(x, y, z, I_(e)) is then analogously obtained which result in a scanningpoint curve in an x-y-z coordinate system.

In the case of a finer scanning, i.e., for example, smaller stepsbetween the individual positions (x,y) of the flat mirrors, the curveresults in an increasingly sharper two-dimensional or three-dimensional“picture” of the part of the eye on which the light beams werereflected.

Concerning 4.2. Detection

4.2.1 Rough Determination of the Pupillary Center

A characteristic feature of the eye is the pupillary center which, forexample, together with the fovea centralis, defines a direction ofvision of the eye. By means of FIG. 8, a rough determination of thepupillary center with respect to the device 420A according to thepresent invention is explained. FIG. 8 is a top view of the image of aneye 480 with the sclera 428, the iris 485 and the pupil 484 behind aspectacle frame 420A.

A plane coordinate system (MS, x, y) is defined for the spectacle frame420A, with respect to which coordinate system the pupillary center(x_(PMG), y_(PMG)) is determined. In this case, a marker MS on thespectacle frame is selected as the origin, the horizontal x-axis and they-axis perpendicular thereto extending through the marker. The x-axis isadvantageously situated approximately at the level of the pupil.

First, as a result of a corresponding movement of the detector mirrors(see “Optical Devices” section), a sequence of points along a firststraight line BM1 on the x-axis is actively or passively scanned. Thebrightness SPH of the scanning elements along this straight line changesat the transition of the scanning beam from the face to the sclera atpoint P1 to a higher value W (“white”), at the transition from thesclera to the iris at point P2 to a lower value and at anothertransition from the iris to the sclera at point P3 back to value W. Thegeometric mean of the x-coordinates of points P2 and P3 is defined asthe preliminary x-coordinate x_(JM) of the pupillary center.

Subsequently, a sequence of points along a second straight line BM2,which extends parallel to the y-axis through the point (x_(JM),0) andthus always through the pupil, is actively or passively scanned. Thescanning beam preferably changes along a three-quarter circle from thestraight line BM1 to the straight line BM2 in order to minimize theoscillation excitation of the device. The brightness SPV of the scanningelements along the second straight line BM2 changes during thetransition from the iris to the approximately black pupil at point P5approximately to a lower value S characteristic of an individual pupiland, during another transition from the pupil to the iris at point P6,again to a higher value. The geometric mean between the y-coordinates ofpoints P5 and P6, between which the scanning beam supplies approximatelya, for example, empirically determined characteristic brightness valueS, is determined as the y-coordinate y_(PMG) of the pupillary center.

Finally, a sequence of points along a third straight line BM3 isactively or passively scanned, which straight line extends parallel tothe x-axis through the point (0,y_(PMG)) and thus always through thepupil. The scanning beam preferably changes along a three-quarter circlefrom the straight line BM2 to the straight line BM3 in order to minimizethe oscillation excitation of the device. The brightness of the scanningelements along the third straight line BM3 changes analogously to thesecond straight line during the transition from the iris to theapproximately black pupil approximately to a lower value S and, duringanother transition from the pupil to the iris, again to a higher value.The geometric mean between the x-coordinates of the points, betweenwhich the scanning beam supplies approximately the brightness value S,is determined as the x-coordinate x_(PMG) of the pupillary center.

By means of above-described methods, an appropriately designed analysisdevice reliably determines the position (x_(PMG),y_(PMG)) of thepupillary center with respect to the spectacle frame 420A and storesthese data, so that, for example, the methods described in the followingcan be used starting from the found pupillary center by which thescanning device is correspondingly readjusted (? There seems to be aword missing in the German.)

Should the expected brightness courses not be recorded during thescanning along one of the three straight lines BM1, BM2 or BM3 because,for example, the eye was closed by the lid, the analysis device willrecognize that the determination has failed and will repeats at leastthe failed scanning steps until a faultless determination of thepupillary center has been carried out successfully and/or will reportthe failing of the determination, from which it can, for example, beconcluded, that a blinking is taking place. It can therebysimultaneously be ensured that the eye is open at the point in time ofthe determination of the pupillary center and, for example, informationcan also be projected into the eye or the retina can be scanned.

In addition, for example, the distance of point P2 to the detector canbe determined and thereby approximately the three-dimensional positionof the pupillary center relative to the device according to theinvention or to the detector can be determined.

4.2.2 Precision Determination of the Pupillary Center

A precision determination of the pupillary center can be carried outwhich will be schematically explained in the following by means of FIG.9. This precision determination can advantageously be carried out afterthe position of the pupillary center has been approximately determinedby means of the above-described rough determination.

FIG. 9 is a top view of an eye with a horizontal line H and a verticalline V perpendicular thereto, which both extend through the (exact)pupillary center PM.

During an active or passive scanning along these lines, approximatelythe brightness sequences SPH illustrated on the bottom or on the rightwith respect to the horizontal line H or SPV with respect to thevertical line V are obtained. In this case, the brightness valuedecreases during the transition from the white sclera to the colorediris from a value W characteristic of the sclera and falls during thetransition from the iris to the black pupil at points P7, P8 or P5, P6approximately to a value S characteristic of the pupil.

When the pupil is now successively scanned along concentric circles K1,K2, . . . Kn around the previously roughly determined pupillary centerPMG, the scanning along the circles which are situated completely insidethe pupil always results approximately in a constant brightness value Sover the entire circumference of the circle; the scanning along thecircles which are situated completely outside the pupil always resultsapproximately in a constant brightness value W along the entirecircumference of the circle.

The radius r1 of a first circle K1 around the roughly determinedpupillary center PMG is selected such that the circle K1 is situatedcompletely within the pupil. For this purpose, the starting value forthe radius r1 is used as the beginning, and the brightness values alongthe pertaining starting circle are compared with the reference value S.Subsequently, the radius is reduced until the brightness values at nopoint significantly deviate from S. As an alternative, an alwayssufficient constant maximum radius can also be defined. Then, concentriccircles K2, . . . Kn around the roughly determined pupillary center arescanned, the radius ri of each circle Ki being greater than the radiusr(i−1) of the preceding circle K(i−1)—until the brightness values alonga circle Kn between points PAUS and PEIN on the circumference of thecircle Kn are significantly greater than the reference value S. In thiscase, the roughly determined pupillary center PMG is displaced along themid-perpendicular onto the secant through points PAUS and PEIN, forexample, by the difference of the radii rn and r(n−1). If the pointsPAUS and PEIN are (approximately) identical, the pupil is completelysituated inside the circle Kn. Then the radius rn is reduced until, atleast on a partial area of the circumference of the circle Kn, thebrightness values no longer significantly deviate from the referencevalue S, and the process is continued.

The above-described method can be repeated several times, the differenceof the radii of successive circles, in contrast to the preceding processpass each being appropriately reduced and a circle with the radius ofcircle K(n−1) being selected as the first circle K1, which was stillcompletely inside the pupil in order to thereby displace the pupillarycenter PMG iteratively into the exact pupillary center PM. When in thiscase, the difference of the radii falls below a predetermined value, thedevice according to the invention stops the process and considers thefound pupillary center PMG as being determined sufficiently exactly.

In the case of the above-indicated method, ellipses can be scannedinstead of circles in order to take into account the distortion of theround pupil in the case of a distortion of the eye. Analogous to theabove method, a first circle which is situated completely outside thepupil can be used as the beginning and the radii of the followingcircles are constantly reduced.

Analogously to the above method, the macula on the retina can also bedetermined. In the following, this will be described by means of FIGS.10 and 11 respectively. In this case, first a pupillary centerdetermined according to one of the above methods can be assumed to bethe starting point for the macula center.

4.2.3 Rough Determination of the Center of the Macula

FIG. 10 is a view of the retina with the macula 686A, the foveacentralis 686 situated therein with the macula center MM, the blind spot688 as well as several blood vessels 687A. The above-mentionedcharacteristics differ from one another and from the remainder of theretina by the brightness values of light beams reflected on them. Thus,for example, the fovea centralis 686 clearly reflects more than theremaining retina; the brightness values of light beams reflected on thefovea centralis are significantly closer to the brightness value W of awhite reflection surface than those of light beams which are reflectedin its environment and therefore have a brightness value situated closerto the brightness value S of a black reflection surface.

In an embodiment of the present invention, the retina is scanned along ahorizontal line H and a vertical line V, which extend through apreliminary macula center obtained, for example, from the previouslydetermined pupillary center in that, for example, the pupillary centeritself is defined as a preliminary macula center, or the preliminarymacula center is defined in a predetermined position with respect to thepupillary center, which may be determined, for example, empirically frompreceding examinations of the relative positions of the pupillary andmacula centers. Then, by means of the position of points P9, P10 or P11,P12, in which a significant jump occurs in the case of the vertical orhorizontal scanning in the brightness course SPV or SPH, and/or by meansof the distance BV or BH between these points analogous to theabove-described methods for determining the pupillary center, the centerof the fovea centralis 686 or of the macula center 686A are determined.

In addition, for example, the distance of points P9, P10, P11 or P12from the detector can be determined, and the three-dimensional positionof the macula center relative to the device according to the inventionor the detector and/or relative to the pupillary center can therebyapproximately be determined. From the three-dimensional position of thepupillary and macula center, for example, the three-dimensional positionof the direction of vision, for example, relative to the detector, canthen be determined.

4.2.4 Precision Determination of the Macula Center

By means of FIG. 11, a method for the precision determination of themacula center is described in detail analogously to the above-indicatedmethod for the precision determination of the pupillary center.Beginning with the previously found pupillary center PM, which is usedas the starting point, the retina is scanned in the region of the macula686 in concentric circles AK1, AK2, . . . Akn, in which case the radiusof a circle Aki is always larger than the radius of the preceding circleAk(i−1). As soon as the brightness values detected along a circle (inFIG. 6A, circle AK5) in an area between a point P13 and a point P14 onthe circumference of the circle have a clearly different value, thepreliminary macula center PM will be displaced along themid-perpendicular to the secant determined by P13, P14 into the newpreliminary macula center P15. The process is repeated with a reduceddifference between the radii of successive circles, starting with thecircle that was still completely inside the macula, i.e., in the case ofwhich the brightness values along the entire circumference of the circleapproximately have a value characteristic of the macula and, forexample, empirically determined and have no significant jumps. Theprocess will be repeated until the difference between the radii ofsuccessive circles falls below a predefined threshold value or a circleextends approximately along the boundary of the macula, which isdetermined in that the brightness values along this circle have aplurality of signal jumps between the value characteristic of the maculaand a value that is clearly above it. The macula center that was foundlast will be stored.

Concerning 4.2.5 Multi-Step and Analogous Methods

It was explained above how, by means of the reflected light beams, thepupillary and/or macula center can be roughly or precisely determined.Likewise, other significant characteristics of the eye, such as bloodvessels of the retina, scars on the cornea or the lie, can also bedetermined. For this purpose, a scanned reflex image, for example, canbe analyzed by means of an image or pattern identification andsignificant patterns (such as blood vessels, scars or the like) can belocated.

Concerning 4.3.2 Determination of the Orientation of the Retina ReflexImage (Retina Map)

FIG. 12 shows a two-dimensional image of the retina of an eye having ablind spot 1188, the macula center MMN and diverse blood vessels. Theretina is scanned by means of a suitable movement pattern, for example,as illustrated in FIG. 12 along a spiral 1138. In this case, asdescribed above, the scanning pattern may be oriented according to theold macula center MMA as the reference point, which had been determinedand stored in an earlier step. While spirals are used for the purpose ofan explanation in the example, the scanning may also take place alongconcentric circles, along a rectangle grid or the like.

The macula center as a reference point may be determined by means of oneof the above-described methods or by means of the present method, whichhad been carried out at an earlier point in time. Starting at the oldmacula center MMA, the detector of the device according to the inventiondetects the brightness values of the light beams which are reflectedalong the spiral. In this case, points SPA of the curve, which aresituated on a blood vessel, have different brightness values than pointson the remainder of the retina; the same applies to points SPAS whichare situated on the blind spot 1188 or to points MMN which are situatedon the macula. The thus obtained brightness values are stored as aretina map.

FIG. 13 shows a thus determined retina map where the brightness valuesare stored, for example, in the binary form; i.e., each point whosebrightness value exceeds a certain characteristic value is provided witha 1, otherwise, it is provided with a 0. A retina map can expediently bestored in the form of a matrix.

By comparing the thus determined new retina map with an old retina mapdetermined at an earlier point in time, corresponding software candetermine the displacement and rotation of the new retina map withrespect to the old retina map and thereby determine the orientationchange of the eye. For this purpose, preferably first by means of one ofthe above-described methods, the new, i.e., actual macula center MMN isdetermined whose position relative to the old macula center MMA resultsin the displacement of the new retina map with respect to the old one;i.e., the pertaining displacement vector W is the vector of MMA to MMN.Subsequently, two preferably orthogonal axes XAN and YAN respectivelyare determined in the new retina map, which intersect with one anotherin the new macula center MMN and are oriented by means of certaincharacteristic features of the scanned points of the retina. Forexample, the axis XAN may extend through points which, because of theirarrangement or their brightness values, are recognized as part of theblind spot. Subsequently, the old retina map is displaced by the vectorW, so that the old macula center MMA is situated on the new maculacenter MMN and is subsequently rotated about this point MMA=MMN suchthat the old axes XAA and YAA respectively will be situated on the newaxes XAN and YAN respectively. The angle of rotation occurring in thiscase, together with the displacement vector W, describes the positionand orientation change of the eye relative to the device with respect tothe position when the old retina map was taken.

Likewise, as illustrated in FIG. 14, a corresponding patternidentification and processing software can also directly make an actual,i.e., newly scanned retina reflex image or an actual retina mapcongruent with a stored old retina map or a retina reflex image. Therotation and/or displacement carried out in the process again describesthe position and orientation change of the eye with respect to theposition which corresponds to the old retina map or the retina refleximage. In this case, it is pointed out that an actual retina refleximage may advantageously be scanned or stored with clearly fewerscanning elements (points). As a rule, these will nevertheless besufficient for making the actual image and the reference imagecongruent, in which case, advantageously fewer points have to be scannedor processed.

In addition, for example, the distance of a point on the retina from thedetector can be detected, and approximately the three-dimensionalposition of the retina and therefore of the eye relative to the deviceaccording to the invention or to the detector can be determined.

Likewise, as illustrated above, the distortion of the reflex image, forexample, of the reflex image of the macula or of the retina, can be usedfor determining the orientation of the eye, as illustrated in FIG. 15 bymeans of a very simplified model. FIG. 15 is a perspective view of aneye 1320 which is rotated by the angle 1310 about the axis 1300.

In the non-rotated condition, a device according to the inventiondetermines an image 1360 of the pupil by means of light beams 1350reflected on the iris. In the rotated condition, the light beams 1350′reflected on the iris in the detector 1340 supply the displaced anddistorted image 1360′ of the pupil 1330′ of the rotated eye. From thedisplacement and distortion of the image 1360′ in comparison with image1360, the rotation 1310 of the eye 1320 can be determinedcorrespondingly. In the knowledge of the actual physical shape, forexample, the circularity of the pupil, the orientation relative to thedetector can also be determined from the distorted image. The actualphysical shape may have been determined, for example, empirically.

In addition, for example, the distance of a point on the iris to thedetector can be determined, and approximately the three-dimensionalposition of the iris or of the pupil and thus of the eye relative to thedevice according to the invention or to the detector can thereby bedetermined.

Concerning 4.3.3 Orientation with Respect to the Environment and/orDevice

In the following, different possibilities according to the invention fordetermining the orientation of the eye relative to the environment orthe device are explained by means of FIGS. 16, 16A-16E. For the purposeof improved clarity, the representation and description is limited tothe simplified plane case. The methods can analogously also be appliedto the generally spatial case. In addition, the distance of one or morescanned points of the eye can be determined, for example, relative tothe detector in that, for example, a frequency-modulated light signal isbeamed onto this point and the reflected light beam is detected. Fromthe frequency of the detected light beam, the point in time of itsemission and thereby its propagation time can be determined, from which,in turn, a distance from the detector or from the projection system canbe determined. The three-dimensional position of the eye relative to thedevice according to the invention can thereby also be determined.

FIG. 16 schematically shows an eye 200 in a reference position and in anorientation (indicated by a broken line and marked by) rotated incomparison with the reference position by the angle α with respect tothe environment, characterized in each case by the axis 210 and 210′ ofvision respectively which extends through the centers of the pupil 220and 220′ and of the macula 230 and 230′ respectively. In addition, theblind spot 240 and 240′ respectively is marked on the retina of the eye200. The environment 250 is outlined here by an incrementalblack-and-white pattern. In an embodiment of the present invention, theenvironment 250 can be photographed by a camera (not shown) which isrigidly connected with the device according to the invention and, in apreferred embodiment, is arranged approximately confocally with respectto the eye 200.

In the reference position, a device according to the invention may, forexample, scan the reflex image of the environment 250 (environmentreflex image, passage scanning) illustrated in FIG. 16A. In addition oras an alternative, a device according to the invention may also activelyscan a portion of the retina of the eye 200. FIG. 16B schematicallyshows the corresponding image with the macula 230A and the blind spot240A (retina reflex image, active scanning). When the light beamed infor the active scanning is appropriately modulated, the environmentreflex image and the retina reflex image can be created separately fromone another. For example, for the active scanning, infrared light isadditionally beamed in during a certain time period T1, while, in asubsequent time period T2, no light beams in actively. A subtraction ofthe image of the retina taken during the period T2 from the image of theretina taken during period T1 approximately supplies the retina refleximage, while image taken during period T2 represents the environmentreflex image.

While the environment reflex image shows structures (edges, etc.) whichare fixed relative to the environment (environment image), thestructures in the retina reflex image are fixed relative to the eye (eyeimage).

By means if a suitable image identification software, both images inFIGS. 16A and 16B can be compared with one another and, for example, thedistance A1 between a significant edge in the reflex image of theenvironment and the macula center can be determined. Analogously, alsoan image of the environment (environment image) taken by the camera andthe image of the retina can be compared with one another and a distancecan also be determined between a significant edge and the macula center.

Orientation of the Eye with Respect to the Environment

By means of the relative position of significant characteristics in theenvironment image 16A and the eye image 16B, outlined, for example, bythe distance A1 between a significant edge and the macula center, anorientation of the eye with respect to the environment can bedetermined. For example, it may be determined that the direction ofvision of the eye points to the right edge of the highest black bar inthe environment.

Change of Orientation of the Eye with Respect to the Environment

When the orientation of the eye 200 changes with respect to theenvironment such that the visual axis 210 changes by the angle α intothe new visual axis 210′ and, in the process, the device remainsconstant relative to the environment, the scanning of the environmentreflex image shows the image illustrated in FIG. 16C which, in theillustrated example, with the exception of the distortion because of thetwo-dimensional imaging of the three-dimensional structure, is identicalwith that of FIG. 16A. A scanning of the retina supplies the retinareflex image illustrated in FIG. 16D with the macula 230A and the blindspot 240A. A comparison of this image of the retina with the refleximage of the environment (FIG. 16C) or the picture of the environment(FIG. 16) taken by the camera again supplies a value A2 of the distancebetween a significant edge in the image of the environment and themacula center. From the change of this distance A1 to A2, the rotation aof the eye with respect to the environment can be determined.

Change of Orientation of the Device with Respect to the Environment

When the orientation of the eye does not change with respect to thedevice, but the orientation of the device does so with respect to theenvironment that the visual axis 210 is again rotated by the angle αinto the axes of vision 201′, a detection of the image of theenvironment reflected on the retina and/or of an image of theenvironment taken by the camera supplies the environment reflex imageillustrated in FIG. 16E. From a comparison of significant edges, animage correlation or the like between this image and the first takenenvironment reflex image from FIG. 16A, a rotation of the device withrespect to the environment can then be determined analogously.

A scanning of the retina, in turn, supplies the retina reflex imageillustrated in FIG. 16B with the macula 230A and the blind spot 240A,from which it can be concluded that the position of the eye relative tothe device has not changed.

As described above, the change of the orientation of the eye withrespect to the environment is composed from the orientation change ofthe eye with respect to the device and the orientation change of thedevice with respect to the environment. Their determination was in eachcase illustrated individually. In the case of a simultaneous change ofthe orientation of the device and the eye, both effects are superimposedon one another and can be determined by a corresponding linking of bothmethods.

On the whole, different possibilities exist in the above-describedexample for determining the orientation of the eye, which will be listedin the following. In the case of a device according to the invention anda method according to the invention respectively, several of thesepossibilities can be combined in order to simultaneously determineseveral relative orientations or check results and, as required, correctthem. As initially mentioned, the variants are explained by means of asimplified plane representation; in the general spatial case,corresponding methods should be used analogously. In addition, thethree-dimensional position of the eye can be determined, for example, bymeans of a distance measurement between a part of the eye and thedetector.

Camera Picture—Retina Image

When the picture of the environment taken by a camera fixedly connectedwith the device (FIG. 16) is compared with the image of the retina whichis, for example, the result of an active scanning (FIGS. 16B, 16D),significant characteristics in the environment image (edges, etc.) andin the retina image (macula, etc.) can be assigned to one another. Bymeans of this relative position (for example, distance A1), anorientation of the eye relative to the environment can be determined.From the change of the relative position of these characteristics withrespect to one another (distance A1, A2, etc.), the change of theorientation of the eye with respect to the environment can bedetermined. Likewise, the two images also can be made congruent (forexample, by a correlation method or by rotation and displacement). Fromthe resulting values (for example, correlation factors or angle ofrotation and displacement), in turn, the orientation change of the eyewith respect to the environment can be determined.

Camera Picture/Environment Reflex Image Structure

From the change of the picture of the environment which was taken by acamera fixedly connected with a device or which is reflected by theretina and detected by a device according to the invention, the changeof the orientation of the device with respect to the environment can bedetermined. For this purpose, for example, two environment reflex images(FIGS. 16C, 16E) are compared with one another which were taken at atime interval from one another. When, in the interim, the orientation ofthe device with respect to the environment has changed by the angle α,significant patterns (for example, continuous vertical bars) have adistance A3 from one another, from which the orientation change a isobtained. Likewise, the two images may also be made congruent (forexample, by a correlation method or by rotation and displacement). Fromthe resulting values (for example, correlation factors or angle ofrotation and displacement), the orientation change of the eye withrespect to the environment can be determined.

Orientation Eye—Device

According to the invention, a marker may be present on a spectacle lensin front of the eye or may be arranged in any other suitable form in afixed position relative to the device in front of the eye.

For this purpose, FIG. 17A is a simplified schematic planerepresentation in the manner of an example of an eye 200 with the visualaxis 210 and the macula 230. A spectacle lens 260, which has a marker261 in the field of vision of the eye, is arranged in front of the eye.FIG. 17B shows an image of the environment, which is reflected by theretina and in which the marker 261 can be recognized (environment refleximage), and is superimposed on an image of the retina reflex with themacula 230.

By means of the relative position of the retina reflex image (eye image)with respect to the marker in the environment reflex image (deviceimage), indicated by the distance A4 between the macula center and asignificant edge of the marker, an orientation of the eye relative tothe device can be determined. An image fixed with respect to the device,such as the image of the marker 261, can be used as the device image.

Likewise, just as a result of the relative position of the eye image,indicated, for example, by the macula center within the image of FIG.17B, an orientation of the eye with respect to a device-fixed coordinatesystem, given, for example, by the left or lower image edge in FIG. 17Bcan be determined, without requiring a marker.

FIG. 17C also shows the eye 200, whose orientation, characterized by thevisual axis 210, changes with respect to the spectacle lens 260 by theangle α. FIG. 17D, again in a superimposed manner, shows an image of theenvironment, which is reflected by the retina and has the marker 261,and an image of the retina with the macula 230. In this case, the imageof the retina can be detected, for example, by means of an activescanning.

A comparison of the two images 17B and 17C indicates that the spacing A4of the images of the marker 261 and of the macula 230 changescorresponding to the orientation change a of the eye relative to thedevice. From the change of this spacing, the change of the orientationcan therefore be determined. As mentioned above, instead of the marking261, for example, the geometrical shape of the spectacle edge itself canalso be chosen as the reference.

Likewise, also without a marker, the orientation change of the eyerelative to the device can be determined just from the change of therelative position of the macula (eye image) within the device-fixedcoordinate system (again given by the left or lower edge in FIG. 17D).

Different possibilities according to the invention were explained aboveas to how, by means of light beams, which are reflected by a part of aneye, characteristic features of the eye can be determined and as to howthey can be used for determining the orientation of the eye relative tothe environment and/or relative to the device.

After several embodiments of the different features of a deviceaccording to the invention and of a method according to the inventionwere described in detail above, which, in an embodiment of the presentinvention, can appropriately be combined with one another, in thefollowing, several embodiments of the present invention will bedescribed in detail as examples.

As initially illustrated, the knowledge of the momentary orientation ofan eye offers a plurality of possibilities:

5.1 Medicine

In the field of medicine, a device for treating the eye by a devicewhich receives the momentary orientation or change of orientation of aneye from a device according to the invention and appropriately processesthese signals (for example, within a position control during whichorientation changes are used as a deviation), can compensate voluntaryand mainly involuntary eye movements (for example, so-called saccademovements or microtremors), so that, for example, a laser beam for thetreatment of the eye or a light signal for examining the eye (isapplied?) in a stationary manner or is guided on a fixed trajectoryrelative to the eye.

5.2 Psychology, Neurology

In the fields of neurology or psychology, certain eye movements, i.e.,orientation changes, can be used as diagnostic aids, for example, forthe prediction of an epileptic seizure, of a blackout, for diagnosingschizophrenia, or the like.

Significantly more extensive applications are opening up if, inparticular, the direction of vision of the eye is determined: Inpsychology, for example, identification patterns when viewing certainimages can be detected and analyzed.

5.3 Direction of Vision

The knowledge of the direction of vision makes it possible to determinein which direction the user is looking. This can be used for identifyinga (possibly only virtual) object on which the user is focusing. For thispurpose, for example, an environment reflex image of the retina can bescanned. When, for example, by means of an active scanning, the positionof the pupillary center or of the macula center is determined withinthis environment reflex image, this point can be determined as the pointat which the user is just looking. An image identification can, forexample, identify the object therefrom at which the user is justlooking.

In this case, the user can indicate, for example, by blinking, keypressure or the like, that he is focusing on the selected object.

Likewise, it can also be determined after a certain time period in whichno eye movements or only slight eye movements have occurred that theuser is focusing on an object and this object can be determined from thedetermined direction of vision.

Likewise, the user can also be instructed to focus on a certain point orobject (which under certain circumstances may also be moved relative tothe user). While the user is focusing, an orientation of his eye may becarried out simultaneously. For example, the position of the pupillarycenter can be determined in each case. From the correlation of thefocused object and the respectively determined orientation, acorrelation can then, for example, be determined between the orientationof the eye and a visual axis, which may, for example, be determined bythe straight connecting line from the focused point to the pupillary ormacula center. By means of this correlation, the orientation of thevisual axis relative to the device can be determined from a determinedorientation of the eye (for example, the position of the pupillarycenter).

5.4 Focus of the User

A suitable device, which receives the signal as to on which image cutoutthe user is just focusing and possibly identifies this image cutout bymeans of image identification, can process this information and triggera desired action:

A launcher can select a desired target in this manner whose coordinatesare used, for example, as target data of a rocket missile;

a user (for example, in the household or in a production operation) canselect a device (such as a light switch, an appliance or a machine tool)that is to be activated, deactivated or controlled in another manner. Acontrol device receives the signal as to where the viewer is looking;identifies the device or the switch at which he is looking, and controlsthe selected device corresponding to a predetermined control program.The user, for example, looks at the system switch of the television;blinks several times (in order to exclude involuntary blinking) and thedevice switches on the television. Then the viewer looks at a certainprogram field, and the device switches the television to the assignedprogram, which again was indicated by repeated blinking, etc.

The patterns on which the user is to focus do not necessarily reallyhave to exist, they may be present only virtually. For example, acomputer screen displays different fields which each correspond to amenu item to be selected, or such fields are projected directly on theuser's retina by the device.

5.5 Projection

For the projection of information onto the retina of the eye, asdisclosed, for example, in PCT Applications PCT/EP00/09843,PCT/EP00/09840, PCT/EP00/09841, PCT/EP00/09842 and PCT/EP01/05886 by theapplicant, it is necessary or advantageous to correlate the (image)information merged into the eye with direction of vision of the eye:

When the information is projected in additionally to the environmentimage which the user perceives, for example, by means ofsemitransparently metal-coated spectacles (see the “Optical DeviceSection”), it may be advantageous to correlate the information with theenvironment image perceived by the retina in such a manner that themerged-in information appears stationary relative to the environment.For example, for this purpose, the image of the environment reflected bythe retina can be detected in the detector, can be appropriatelyprocessed and, while utilizing the determined orientation of the eye,can be congruently with the actually perceived environment imageprojected back into the eye.

In addition, for example, by means of a camera, a picture can be takenof the environment, can be processed in an appropriate manner and, whileutilizing the determined orientation of the eye, can be projected intothe eye such that it appears congruent with the actually perceivedimage. As a result, it becomes possible to take an infrared image of theenvironment, for example, by means of a camera and to merge theresulting environment image as additional information into the eye inorder to improve the vision in fog, at night or under other difficultvisual conditions.

Likewise, the camera may have a binocular lens system for enlargingremote images. Then, for example, in the center of the environment imageperceived by the eye, an enlarged image detail can be projected into theeye which, compared to conventional binoculars, may have the advantagethat the marginal areas of the environment image are not enlarged andtherefore the orientation of the user with respect to the environment isnot impaired or not as significantly impaired. As an alternative, acomplete enlarged image of the environment can also be projected intothe eye which covers the naturally perceived environment image, in whichcase small relative movements of the device, which result, for example,from the natural muscular tremors or shocks during the car drive, arecompensated such that the “binocular” image appears stationary to theuser. Such a binocular device avoids the micro movements in the case ofconventional binoculars which are perceived to be disturbing.

Likewise, it becomes possible by means of determining the orientation ofthe eye to project information into the eye such that it moves relativeto the environment or appears to be stationary relative to a movingobject. For example, the distance to a vehicle driving ahead could bemeasured and projected into the driver's eye in such a manner that theinformation seems fixed relative to the vehicle driving ahead. Thedriver thereby receives necessary information without having to lookaway from the vehicle driving ahead. Inversely, for example, a text tobe read can be projected into the eye such that it seems to move infront of the viewer's focus (i.e., the area of the perceived image atwhich the interest is aimed) corresponding to a predetermined movementpattern (for example, uniformly from the right to the left, discretelyas groups of words or letters, etc.). Likewise, a text to be read can beprojected into the eye such that it seems to be stationary with respectto the environment in that, for example, the determined orientationchanges are taken into account during the projection of the text.

When, in contrast, the information is projected into the eye without anysimultaneous natural perception of the environment, as it occurs, forexample, in the case of virtual-reality glasses where the user perceivesonly the images projected into the eye, the determination of thedirection of vision of the eye makes it possible to adapt theprojected-in information, for example, the view of a virtual landscape,to the user's direction of vision. Thus, when the viewer looks to theleft by rotating his eye relative to the glasses, a device according tothe invention will recognize this orientation change and willcorrespondingly change the apparent angle of view with respect to thevirtual landscape.

In the following, the essential points are summarized again by means ofgroups of characteristics which, each separately and in combination withone another, characterize the present invention in a special manner.

1. Device according to the invention or a method according to theinvention for determining the position and/or the orientation,particularly the direction of vision, of an eye, a starting point or anend point of a light beam reflected by a part of the eye and detected bya detector system and/or of a light beam projected by a projectionsystem onto or into the eye quasi two-dimensionally describes a movementpattern of a scanning and/or projection movement in the eye when thedirection of the light beam is changed with respect to the timeaccording to the scanning or projection movement.

2. Device according to Point 1, having

-   -   a shifting device which causes a reference point of the movement        pattern to follow into the pupillary or macula center, and    -   a determination device which uses the movement pattern of the        scanning movement or projection movement for determining the        pupillary center or the macula center.

3. Device according to Point 1 or 2, wherein the determination of thedirection of vision of the eye takes place without being perceived.

4. Device according to one of the preceding claims, wherein the deviceis constructed to be wearable (portable), particularly in the form ofspectacles.

5. Device according to one of the preceding points, wherein theprojection system is projected (projects?) an infrared light beam ontothe eye whose diameter is very small in comparison to the pupillarydiameter, and the ocular, particularly the retinal reflex of the beam isdetected.

6. Device according to Point 5, wherein the infrared light beam has adiameter of less than 100 μm at the air-cornea transition.

7. Device according to one of Points 5 to 6, wherein the infrared lightbeam has a diameter of less than 50 μm at the air-cornea transition;

8. Device according to one of Points 5 to 7, wherein the infrared lightbeam has a diameter of less than 10 μm at the air-cornea transition.

9. Device according to one of claims 5 to 8, wherein the infrared lightbeam has a diameter of less than 5 μm at the air-cornea transition.

10. Device according to one of claims 5 to 9, having a splitter mirrorarranged in the infrared light beam, which splitter mirror allows only asmall fraction of the infrared light beam to pass and reflects acorrespondingly large fraction of the incident ocular reflex in thedirection of a detector device.

11. Device according to one of the preceding points, wherein theprojection system projects light in a pixel-type manner with apredefined pixel frequency onto the eye.

12. Device according to Point 11, wherein the projection systemmodulates the projected light with a frequency that is higher than thepixel frequency.

13. Device according to Point 11 or 12, wherein the projection systemmodulates the projected light with a frequency that is a multiple of thepixel frequency.

14. Device according to Point 13 which carries out the modulation suchthat detected projected light reflected back from the eye can bedifferentiated from the ambient light.

15. Device according to one of the preceding claims, wherein no activeillumination of the eye takes place, and the detector system carries outa pixel-type scanning of the ambient light reflected back from the eyeand/or of the light emitted by the eye.

16. Device according to one of the preceding points which illuminatesthe eye in a surface-type manner by infrared light, and wherein thedetector system carries out a pixel-type scanning of the infrared lightreflected back from the eye.

17. Device according to claim 15 or 16, wherein the detector system isdesigned for scanning pixels of a size of less than 100 μm2 of theretina.

18. Device according to one of Points 15 to 17, wherein the detectorsystem is designed for scanning pixels of a size of less than 25 μm ofthe retina.

19. Device according to one of the preceding points, having a surfacethat can be positioned in front of the eye, which has marker areas whichreflect an incident projection beam originating from the projectionsystem completely back in the direction of the detector system, as wellas normal areas which guide an incident projection beam originating fromthe projection system in the direction of the center of the eye.

20. Device according to one of the preceding points which determines theposition and/or orientation of the eye with respect to its environmentin that the detector system detects the retinal structure of the eye aswell as the environment reflex image superimposed thereon, determinesthe position of the fovea by means of the retina structure andidentifies the area of the environment sighted by the fovea by means ofa pattern identification.

21. Device according to one of the preceding points, which determinesthe direction of vision of the eye in that it determines the change ofthe relative position between the optical detector and/or projectionsystem and the eye.

22. Device according to one of the preceding points, having a guidingdevice, which uses the information content of the light detected duringthe scanning movement for determining time-related changes of therelative position of the optical detector and/or projection system withrespect to the optical system of the eye in order to cause the movementpattern of the scanning and/or projection movement to follow on thebasis of the determined change of the relative position of thetime-related position changes of the eye.

23. Device according to one of the preceding points, wherein the opticaldetector and/or projection system is a system for the emission ofsignals as a function of image information incident on the human retina.

24. Device according to one of the preceding points, having an analysisdevice, by which the information content, preferably as gray values, ofthe light reflected by the eye and detected by the detector system canbe analyzed in two coordinates.

25. Device according to one of the preceding points, through which themovement pattern of the scanning movement passes several times, at leastin sections, particularly repeatedly, until unambiguous values for thecoordination of the pupillary or macula center are present.

26. Device according to one of the preceding points, which switches astarting pattern in front of the movement pattern of the scanningmovement for determining the pupillary or macula center, which startingpattern, as a result of the analysis of the information content,preferably of the gray values, of the light detected by the detectorsystem in two coordinates, is used for the rough determination of thecoordinates of the pupillary center.

27. Device according to Point 27, having a reference point from whichthe starting pattern originates.

28. Device according to Point 26 or 27 which uses the coordinatesdetermined during the rough determination of the pupillary center whendefining the movement pattern of a subsequent scanning movement for theprecision determination of the pupillary or macula center.

29. Device according to one of Points 27 to 29, wherein the startingpattern for the rough determination of the pupillary center comprises atleast three linear movement sections, the first movement sectionpreferably originating from the reference point, which intersects twicewith a transition between the iris and the sclera of the eye, beingfollowed by a second movement section, which extends along themid-perpendicular of a first secant that corresponds to the firstmovement section between the two iris/sclera transitions, and the thirdmovement section, in turn, standing perpendicularly on the secondmovement section and extending either through the center of the pupildetermined during the second movement section by way of the informationcontent, preferably by way of the gray values, of the detected light,or, in the center, intersecting a second secant formed by the secondmovement section with respect to two iris/sclera transitions.

30. Device according to one of the preceding points which, for theprecision determination of the pupillary center, carries out a scanningmovement in the pattern of a circular spiral or elliptic spiral or ofconcentric circles or ellipses around roughly determining coordinates ofthe pupillary center.

31. Device according to Point 31, wherein previously stored coordinatesof the pupillary center are used as roughly determining coordinates ofthe pupillary center.

32. Device according to Point 31, wherein, roughly determinedinstantaneous coordinates of the pupillary center are used as roughlydetermining coordinates of the pupillary center.

33. Device according to one of Points 31 to 33 which recursively refinesthe roughly determining coordinates of the pupillary center by means ofthe information content, preferably by means of the gray values, of thelight detected during the scanning movement for the precisedetermination of the pupillary center.

34. Device according to one of Points 31 to 34 which terminates thescanning movement for the precision determination of the pupillarycenter when the values, particularly the gray values, of the lightdetected during a cohesive scanning movement section passing through atleast 360°, do not fall outside a predetermined area.

35. Device according to Point 30 which uses the point at which the thirdmovement section crosses an iris/pupil transition for the second time asthe starting point for a scanning movement for the precisiondetermination of the pupillary or macula center.

36. Device according to one of the preceding points which, for theprecision determination of the macula center and/or structure, carriesout and/or repeats a radially increasing scanning movement originatingfrom the coordinates obtained during the determination of the pupillarycenter in the pattern of a circular or elliptic spiral or of concentriccircles or ellipses until the information content, preferably the grayvalues, of the light detected during the radially increasing scanningmovement supplies a clear indication of the diameter and/or the centerof the macula.

37. Device according Point 31 which terminates the scanning movement forthe precision determination of the macula center and/or structure whenthe information content, preferably the gray values, of the lightdetected during a cohesive scanning movement section passing through atleast 360° repeatedly has a clear signal jump from a light value to adark value and vice-versa.

38. Device according to one of the preceding points which determines therelative position of at least one characteristic area of the retina withrespect to the optical detector and/or projection system, and which usesthe deviations of determined position data of this characteristic areafrom previously stored position data of this characteristic area for thedetermination of the spatial position and/or orientation of the eye withrespect to the optical detector and/or projection system.

39. Device according to one of the preceding points which detects arepresentation of at least selected areas of the retina and stores it inan intermediate memory and, for determining a change of the spatialposition of the eye, makes a comparison of the filed representation withinformation which the device has obtained from light scanned from theretina and detected during an actual scanning movement.

40. Device according to one of Points 38 to 40 which uses the iris, thesclera, the cornea and/or another structure of the eye instead of theretina or together with the retina.

41. Device according to Point 39 which, as a characteristic area, usesat least one section of the vessel structure of the retina.

42. Device according to one of the preceding points, wherein light isdetected in the visible and/or in the infrared range by the detectorsystem.

43. Device according to one of the preceding points, having a memorydevice in which the coordinates of the pupillary or macula center withrespect to a reference point on the optical detector and/or projectionsystem can be stored.

44. Device according to one of the preceding points, having alight-guiding arrangement by means of which the beam path of the lightdetected by the detector system and/or projected by the projectionsystem can be controlled corresponding to the movement pattern of thescanning or projection movement, and an adjusting device by means ofwhich a neutral center position of the beam-guiding arrangement can becaused to follow while using the change of the coordinates of thepupillary or macula center.

45. Device according to one of the preceding points, having abeam-guiding arrangement which can control the beam path of the lightdetected by the detector system in such a manner that it describes acircular or elliptic spiral or concentric circles or ellipses in theeye.

46. Device according to Point 46, wherein the position of thebeam-guiding device is used for determining the relative position of atleast one characteristic area of the retina with respect to the opticaldetector and/or projection system.

47. Device according to one of the preceding points, with a surfacehaving a predefined geometrical shape which can be positioned in frontof the eye and by way of which light can be projected from theprojection system into the eye, the geometrical shape of the surfacebeing used for determining the relative position of at least onecharacteristic area of the retina with respect to the optical detectorand/or projection system.

48. Device according to one of the preceding points, having a memorydevice by means of which the rough coordinates of the pupillary centercan be stored corresponding to a rough determination of their position.

49. Device according to one of the preceding points, having

-   -   a device for determining the relative position of at least one        characteristic area of the retina with respect to the optical        detector and/or projection system,    -   a comparator device by means of which the deviations of        determined position data of this characteristic area from        previously stored position data of this characteristic area can        be used for determining the spatial position of the eye with        respect to the optical detector and/or projection system.

50. Device according to one of the preceding points, having areadjusting device by means of which the movement pattern of thescanning and/or projection movement can be readjusted corresponding todeviations of determined position data of at least one characteristicarea of the retina from previously stored position data of thischaracteristic area, in order to shift the center of the movementpattern of the scanning and/or projection movement, which was previouslysituated in the pupillary or macula center, again into the pupillary ormacula center of the eye, and/or in order to cause the movement patternto follow the time-related position changes of the optical system of theeye.

1. (canceled)
 2. A method for determining the position and/ororientation of an eye, comprising: detecting light reflected by at leasta part of the eye using a detector system of a device, determining theposition and/or orientation of the eye relative to an environment atleast partially according to the detected light, determining an objecton which a user is focusing from the position and/or orientation of theeye relative to the environment, and controlling the objectcorresponding to a predetermined control program.
 3. The methodaccording to claim 2, wherein detecting light reflected by at least apart of the eye is based on active scanning and/or passive scanning. 4.The method according to claim 2, wherein the light can first be activelyprojected into the eye by means of a projection system, such as a laser,a radiator, and a beam.
 5. The method according to claim 2, furthercomprising: determining the position of the pupillary center of the eyefrom the detected light, and determining the position and/or orientationof the eye relative to the device from the position of the pupillarycenter of the eye.
 6. The method according to claim 2, furthercomprising: determining the position and/or orientation of the devicerelative to the environment from an orientation determining device. 7.The method according to claim 6, wherein the orientation determiningdevice may comprise a GPS receiver and a pertaining analysis device fordetermining the position of the device from received GPS signals, andwherein the orientation determining device is fixedly connected to thedevice.
 8. The method according to claim 6, wherein the orientationdetermining device may also be fixedly connected with the environmentand determine the orientation of the device, for example, by means oftriangulation or the like.
 9. The method according to claim 2, furthercomprising: capturing a first camera image and at least a second cameraimage using a camera by photographing the environment at a time intervalfrom one another, wherein said camera is rigidly connected with thedevice, and wherein the first camera image and the at least secondcamera image are compared with one another in a common coordinate systemfor determining the position and/or orientation of the device relativeto the environment.
 10. The method according to claim 2, wherein theorientation of the eye with respect to the environment can be calculatedfrom the relative orientation of the eye with respect to the device andthe relative position of the device with respect to the environment. 11.The method according to claim 2, further comprising: detecting at leastone environment reflex image by the detected light, capturing at leastone environment image using a camera by photographing the perceivedenvironment, wherein said camera is rigidly connected with the device,and identifying a significant structure in both the environmental refleximage and the environment image and determining the orientation of theeye relative to the environment from the spatial assignment of thepositions of said significant structure in the environmental refleximage and the environment image.
 12. A device for determining theposition and/or orientation of an eye, comprising: a detector system fordetecting light reflected by at least a part of the eye, the positionand/or orientation of the eye relative to the device being determinedfrom the detected light, an orientation determining device fordetermining the position and/or orientation of the device relative to anenvironment, and a control device for receiving the signal as to where auser is looking, determining an object on which the user is focusingfrom the position and/or orientation of the eye, and controlling theobject corresponding to a predetermined control program.
 13. The methodaccording to claim 12, wherein the orientation determining devicecomprises a GPS receiver and a pertaining analysis device fordetermining the position of the device from received GPS signals, andwherein the orientation determining device is fixedly connected to thedevice.
 14. The method according to claim 12, wherein the orientationdetermining device is also fixedly connected with the environment anddetermines the position and/or orientation of the device, for example,by means of triangulation or the like.
 15. A device for determining theposition and/or orientation of an eye, comprising: a detector system fordetecting light reflected by at least a part of the eye, wherein theposition and/or orientation of the eye relative to the device isdetermined from the detected light, a camera for capturing a firstcamera image and at least a second camera image by photographing theenvironment at a time interval from one another, wherein said camera isrigidly connected with the device, wherein the first camera image andthe at least second camera image are compared with one another in acommon coordinate system for determining the position and/or orientationof the device relative to an environment, and a control device forreceiving the signal as to where a user is looking, determining anobject on which a user is focusing from the position and/or orientationof the eye, and controlling the object corresponding to a predeterminedcontrol program.
 16. A device for determining the position and/ororientation of an eye, comprising: a detector system for detecting atleast one environment reflex image by detecting light reflected by atleast a part of the eye, a camera for capturing at least one environmentimage by photographing the perceived environment, wherein said camera isrigidly connected with the device, a determination device whichidentifies a significant structure in both the environmental refleximage and the environment image and determines the orientation of theeye relative to the environment from the spatial assignment of thepositions of said significant structure in the environmental refleximage and the environment image, and a control device for receiving thesignal as to where a user is looking, determining an object on which auser is focusing from the position and/or orientation of the eye, andcontrolling the object corresponding to a predetermined control program.17. The device according to claim 11, further comprising: a projectionsystem, such as a laser, radiator, and a beam, for actively projectingthe light into the eye.
 18. The method according to claim 2, wherein theuser can indicate, for example by blinking, key pressure or the like,that the user is focusing on the object, or the object can also bedetermined after a certain time period in which no eye movements or onlyslight eye movements have occurred that the user is focusing on anobject.
 19. The method according to claim 2, wherein the object mayreally exist, or may be present only virtually.
 20. The device accordingto claim 12, wherein the device may be constructed in the form ofspectacles, or may be arranged on or integrated in a frame wearable onthe head, or may also be arranged on or integrated in a stationaryapparatus, or may be arranged on or integrated in an object which ismovable relative to the environment and relative to the user.